UPDATE: Bizarrely, a paper exists which is roughly in accordance with the original ethics application:
“In a large experiment, participants extrapolated global climate data, presented graphically either as share prices or temperatures.” (Link)
So it looks (allegedly) as if Lewandowsky, having completed the paper for which the approval was sought, submitted an amendment to that same ethics application in order to then use it a second time for a completely different paper.
As a result of my recent Freedom of Information request to the University of Western Australia, I have received copies of emails between Prof Lewandowsky and various blogs in connection with the “moon landing denier” paper survey (which I will deal with at a later date), and also the correspondence regarding the application to the university’s Ethics Committee (“EC”).
This latter document reveals that the project was substantially altered at the last minute, and the alteration was waved through by the EC in less than 24 hours.
This is a long post, so click through to read.
The original application to the EC was submitted on or about 14 December 2009, the title of the project being (section 1 of the application):
Understanding Statistical Trends
The aims of the project were described thus (section 7):
The project seeks to explore people’s understanding of statistical trends in time series data. If we are monitoring a stock price, what do we think will happen to it in the future?
Participants will be shown simple graphs of time series (samples enclosed) and will make predictions about the future trends.
This clearly bears no relation to the “moon landing denier” survey. Here’s an example of one of the graphs Lewandowsky planned to used:
The procedures are expanded upon in a further part of the application (section 9):
Subjects will be shown a number of statistical graphs (3 or 4 at most) that contain time series data. Subjects will be asked to extrapolate the visible trend into the future by indicating their guess of the next most likely values (see enclosed sample).
Some of the trends will be upward, some downward, and most will be presented as fictitious stock prices. The actual data will either be generated randomly or will be the world’s temperature (climate) data collected by NASA (NASA GISS data set).
For some subjects, the climate data will be identified as such whereas for other subjects (chosen at random) they will be presented as stock prices.
Again, this bears no relation to the final survey. Only at the end of this description do we get a hint of where that may originate:
Upon completion of the graphical task, subjects will be presented with 3 – 4 questions about their impressions of scientific certainty. For example, people will be asked how certain they think sceintists are about the association between emissions and climate change, HIV and AIDS, and tobacco smoke and lung cancer (using a scale from 0 – 100%). For each item the participants will also provide their own subjective certainty rating (i.e. “how certain are [you] that human emissions [HIV] are largely responsible for climate change [AIDS]“).
This section ties in with the questions labelled ConensHIV, ConsensSmoke, ConsensCO2 (in respect of scientific certainty) and CauseHIV, CauseSmoke and the CO2 questions (in respect of subjective rating) in the final survey (here – PDF). However, this is almost an afterthought, something tacked on once the main purpose of the survey was completed – namely the ability of people to extrapolate graphical trends.
The participant group (section 8) is also fundamentally different:
Members of the population at large as well as the UWA campus community will be approached by the researchers and will be asked to complete a brief questionnaire (5 minutes). No remuneration or compensation will be offered.
Also, in discussing the method of approaching participants:
The researcher will approach potential participants in public places (e.g. City of Perth) and ask if they wish to voluneer 5 minutes of their time.
To me this looks like an application for an entirely different survey. One small further point is that in the ethics checklist, one question asks, “Does the research involve active concealment of information from participants and/or planned deception of participants” to which the answer is “NO”. However, section 9 (cited above) shows that some participants will be shown climate data labelled as stock prices. I would have thought that would be regarded as concealing information or planned deception, but I’m not pressing the point.
About a week later, on 21 December 2009, the Ethics Committee approved the application by a letter which included the following, presumably standard, condition:
The application and all supporting documentation form the basis of the ethics approval and you must not depart from the research protocol that has been approved.
No special conditions were imposed.
On 12 August 2010 (Thursday), at 12.07pm (the time is important here), barely a fortnight before the survey was eventually emailed to various blogs, Lewandowsky writes to Kate Kirk, an administrative officer in the Ethics office. She no longer appears in the online directory, but a Google cache provides the necessary information (here - 27 September 2012). The administrative officer in the Human Research Ethics Officer is now Dr Caixia Li. Kate Kirk was apparently still in that role within the last month given the cache date. I won’t speculate on the circumstances surrounding this. UPDATE: Kate Kirk appears on one HREO contact page but not another… odd. She still does not appear in the full staff directory.
The text of his email is as follows:
Dear Kate, I am writing to seek approval for an amendment to the procedure for RA/4/1/4007. In a nutshell, I want to administer the survey not in person but via the internet using professional survey software (e.g. http://www.surveymonkey.com or equivalent.
As before, completion of the survey will be taken to constitute consent, and as before a variant of the approved infomration sheet will be shown before the survey commences.
The survey will be modified slightly as follows,
(1) The graphical extrapolation task is removed
(2) In addition to the already-approved items, some further questions will be presented that are enclosed in this email
[information regarding questions already approved not reproduced here]
(3) In all other respects the approved procedure remains unchanged except that it is administered via internet, with consent again being expressed by completion of the electronic questionnaire.
(4) Participants will be recruited by posting links at relevant websites (e.g. http://www.uwa.edu.au/climatescience or science-oriented “blogs”.).
There are a number of particularly concerning issues which are raised by this email.
- In the first paragraph, “amendment to the procedure” by using internet survey software rather than personal contact is a highly selective representation of the substantial changes proposed;
- Removal of the graphical extrapolation task effectively undermines the primary aim of the project which is clearly set out in Section 7 of the application, namely “to seek peoples understanding statistical trends time series data” and could hardly be characterised as a “slight” modification;
- “Some further questions” (again, written as almost an afterthought) is actually the 40-plus new questions relating to conspiracy theories and free markets, a subset of which comprise the majority of the final survey as posted on the various blogs.
The request contains no amendment to the aims of the project set out in Section 7, nor to the details of the procedures set out in Section 9 (nor even to the project title!), which no longer reflect the nature of the project and the associated survey, but focuses primarily on the method of administering the survey and the inclusion of “some further questions”.
Response from the EC
At 9.31am the next day (Friday), Kate Kirk replies:
Project: Understanding Statistical Trends
Approval is granted for the amendments outlined in your email and attachments.
Even the title she quotes doesn’t make sense any more – yet there was no amendment requested by Lewandowsky. So astonished was Lewandowsky to get a reply so quickly, he wrote back at 9.50am:
Hi Kate – wow, thanks for the quick approval.
He then asks for approval to leave his name out of the online survey, because:
my name alone routinely elicits frothing at the mouth by various people [link to a Jo Nova blog post], not to mention the hate mail I receive.
Kate is fine with this in an email response five minutes later, and jokes that:
I look forward to receiving the hate mail. I’ll let you know if I get any.
It seems clear from the above that the EC approval originally obtained was for a fundamentally different project, and the nature of the amendment and its rapid approval raises a number of questions for the university:
- What form did the EC review of the request for amendment take?
- Was it submitted to the whole committee for reevaluation?
- How was it possible that the EC could have reviewed such substantive changes and come to a decision within 24 hours?
- Why was there no request from the EC to Lewandowsky to reframe the aims and detailed procedures of the project, which now made little sense?
- Why was there no request from the EC for an explanation as to the purpose of the conspiracy/market forces questions, which were not addressed at all in the application (either in original form or as amended)?
No doubt we will learn more in due course.
The application and related correspondence is here – PDF.