Ian Castles

8 August 2008



QUOTE: The report's findings were accepted without question by the
government and by most commentators, despite the fact the report had not
been subjected to any external review.



QUOTE: It is truly unfortunate that the CSIRO and the Bureau of
Meteorology did not undertake a prior evaluation of the models used in
the first study of its kind in Australia in order to ensure that those
models were able to reproduce the past situations that were relevant to
the study.



QUOTE: as the model simulations have no resemblance to observed droughts
in the last century, the models have failed internal validation and no
further testing is warranted;







On July 6, 2008 Australia's Prime Minister, Kevin Rudd, announced the
imminent release of "a new report on the impact of climate change on
drought". The CSIRO and the Bureau of Meteorology had presented the
government with some "very disturbing" findings:



What they say in two short points is this, and this will come out later
today - firstly that when it comes to exceptional or extreme drought,
exceptionally high temperatures, the historical assumption that this
occurred once every 20 years has now been revised down to between every
one and two years, that's the first point.



Secondly, with exceptional circumstances drought conditions, under
scenarios within it, that that will occur twice as often, and with twice
the area of droughted parts of Australia included. Now this is a serious
revision of the impact of climate change on drought and the Agricultural
Minister will make that clear in the report that he releases later today
...



At a press conference some hours later, the Minister for Agriculture,
Fisheries and Forestry, Tony Burke, said that some of the predictions of
drought he had received in the new report "read more like a disaster
novel than a scientific report". The Minister claimed that the
conclusions of the report meant that "we now know what would happen if
we did nothing", and that "we know for certain from today ... that the
ground rules, because of climate change, have themselves changed"
(emphases added).



Asked how confident he could be that predictions based on modelling were
accurate, Mr Burke replied that "What we've done is taken the best
climate scientists in Australia and asked them to come up with their
best information". The scientists had looked at a range of projections,
and "we all hope that we can work our way forward to reducing the
greenhouse gases in the atmosphere and start to ameliorate what some of
these projections are based on". However, "the bottom line ... is that
this has been handed to the government, but it's entirely written from
beginning-to-end by scientists".



In a further press conference on the following day Minister Burke
repeated that he'd taken his information from "the best available
science we can get" and that "At every level, the news is simple: it's
just getting tougher and likely to continue to get tougher in the years
to come". Averring that all of the projections went to the same
conclusion that there would be "varying levels of increased hardship",
Mr Burke reiterated that his "priority on all of this is to make sure we
go with the best available science and I do believe that's what the
CSIRO and the Bureau of Meteorology gave us yesterday".



Predictably, the Minister's "disaster novel" metaphor made headlines in
Australia and overseas.



According to The Guardian (London), the new report predicted that
Australia "will be hit by a 10-fold increase in heatwaves and ...
droughts will almost double in frequency and become more widespread
because of climate change". Together with the Garnaut report published
some days earlier, the CSIRO/Bureau of Meteorology report "would put
pressure on Prime Minister Kevin Rudd to act swiftly on his pledge for
Australia to lead the world in tackling polluters".



The report's findings were accepted without question by the government
and by most commentators, despite the fact the report had not been
subjected to any external review.



However, Dr David Stockwell, author of Niche Modeling and host of the
Niche Modeling website, saw a need to check on the tests that the
authors had conducted to determine the statistical significance of their
conclusions. For this purpose, he asked CSIRO for the results from each
of the 13 models that had been used to produce the drought projections.
To his surprise, this request was initially refused. However, following
extensive canvassing of the issue on several other websites, the data
were eventually released some three weeks after the publication of the
report.



Dr Stockwell has now published the results of his review of the report,
using the detailed information that is available here.



He has found that:



a.. all 13 climate models failed internal validation tests for
regional droughted area in Australia over the past century;


a.. simulations showed increases in droughted area over the last
century in all regions, while the observed trends in drought decreased
in five of the seven regions identified in the CSIRO/Bureau of
Meteorology report;


a.. in almost all cases, the correlation coefficient between simulated
and observed values was very low, and not significant;


a.. in all cases the "coefficient of efficiency" was negative,
indicating that the climate models simulated drought area worse than
simply using the mean;


a.. in almost all cases the difference between the means of the return
periods was significant, indicating that the frequency of droughts in
the models has no relationship to the actual frequency of droughts;


a.. as the model simulations have no resemblance to observed droughts
in the last century, the models have failed internal validation and no
further testing is warranted;
b.. contrary to statements in the report, there is no credible basis
for claims of increasing frequency of Exceptional Circumstances
declarations;


a.. and as there is no logical connection between the extreme values
of models and simulations using different global warming scenarios, the
report's claim to have performed analysis using high global warming
scenarios is illogical and invalid: this is true, irrespective of
whether or not the underlying climate projections are themselves
invalid.


Dr Stockwell recognises that some of the models show apparent skill at
some statistics in some regions, but says that this is of no importance:



For example, the climate model miroc-h from the Centre for Climate
Research, Japan, shows good agreement with observed return period in
most regions. However, it has not been subjected to more rigorous
external validation, and it has performed poorly in other tests ... used
in a study including three co-authors of the DEC report.



In the light of these findings, Dr Stockwell suggests to the authors
that "Studies of complex variables like droughts should be conducted
with statisticians to ensure the protocol meets the objectives of the
study", and offers some examples of how known statistical methods might
be used to improve drought projections. More generally,



Stockwell concludes that "Policy making based on science should not be
influenced by studies that appear scientific but do not adhere to the
generally accepted validation practices".



Dr Stockwell's critique of the CSIRO/Bureau of Meteorology report does
not constitute a rejection of the need for the application of the
scientific method in climate science modelling. On the contrary, his
analysis is in itself an affirmation of the need for a rigorously
scientific approach in this area - and his successful efforts to bring
about the publication of all supporting information will not only
facilitate the external scrutiny of the Drought Exceptional
Circumstances Report that its importance demands, but will also enable
other experts to seek to reproduce Stockwell's analysis. This can only
lead to more enlightened public policy.



It should be stressed that Dr Stockwell's study does not demonstrate
that droughts in Australia will not become more frequent and/or more
severe. But if his analysis withstands expert scrutiny he will have
shown that the report on future drought events by Australia's two
leading climate science research bodies does not meet basic tests of
climate model validity and cannot serve as a guide to policy.



It is truly unfortunate that the CSIRO and the Bureau of Meteorology did
not undertake a prior evaluation of the models used in the first study
of its kind in Australia in order to ensure that those models were able
to reproduce the past situations that were relevant to the study.



The website of the Statistical Society of Australia Inc lists the names
of more than 100 accredited statisticians who wish their contact details
to be made public. There is now an opportunity for those experts who
have the requisite skill and inclination to carry out their own analysis
of the data used in the CSIRO/Bureau of Meteorology study and to report
the results. In doing so, they would be performing a valuable public
service.



http://www.onlineopinion.com.au/view.asp?article=773
6&page=2
--


Warmest Regards

Bonzo


"One thing lefties cannot abide is any hint of Christian religion. New
Age yes, Muslim yes, vague yoga-nistics sure, Mother Earth and soaring
hawks as spiritual messengers absolutely, but Christianity? Not a
chance." Anon