UQ: Threats and hot air – and an FoI

Pacman lives!

UPDATE: Even the Washington Post is now in on this story, with a piece entitled: Is it copyright infringement to post a lawyer’s cease-and-desist letter?

More popcorn required, as UQ jumps the shark on the release of data relating to John Cook’s Consensus Project.

If you google “97%” and “climate” it returns nearly 13 million hits, all thanks to Un-Skeptical Pseudo-Science’s attempt to shut down debate. The Consensus Project used a number of raters, the majority of whom were warmists, to review abstracts from 26,000 papers and categorise them as to their agreement or otherwise with the global warming consensus. The results were then published in a journal, the abstract of which reads:

We analyze the evolution of the scientific consensus on anthropogenic global warming (AGW) in the peer-reviewed scientific literature, examining 11 944 climate abstracts from 1991–2011 matching the topics ‘global climate change’ or ‘global warming’. We find that 66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming. Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming. In a second phase of this study, we invited authors to rate their own papers. Compared to abstract ratings, a smaller percentage of self-rated papers expressed no position on AGW (35.5%). Among self-rated papers expressing a position on AGW, 97.2% endorsed the consensus. For both abstract ratings and authors’ self-ratings, the percentage of endorsements among papers expressing a position on AGW marginally increased over time. Our analysis indicates that the number of papers rejecting the consensus on AGW is a vanishingly small proportion of the published research.

Apart from the obvious subjectivity involved in such an exercise, the authors then made it difficult for anyone to access the data in order to replicate the results. Surprised much? Hey, these are climate scientists after all! As Richard Tol lamented:

Mr Cook has released more data. Unfortunately, a lot of data is still missing. Particularly, rater IDs and time stamps are not available. This means that we cannot test for systematic differences between raters. It also means that we cannot compute average and minimum rating times.

Dan Kammen, the editor of Environmental Research Letters, has explicitly endorsed Cook’s refusal to release all data, against the journal policy. No word yet from the University of Queensland or the Institute of Physics.

Three new data sets were released. Ratings 4a and b are now available. This reveals that they had another look at 1000 papers, re-rated 5, and scaled this up to 40 for the entire sample. That’s fine. Paper IDs were released too. Most importantly, ratings are now there. The data are in the order of rating. The data have the paper ID, the rater’s rate, the rater’s topic, the final rate and the final topic. 

The identity of the raters is public knowledge. They are the authors of the paper, plus several others listed in an acknowledgement at the end. The majority are, suffice it to say, members of the Skeptical Science in-crowd:

• John Cook
• Dana Nuccitelli
• Sarah A Green
• Mark Richardson
• Bärbel Winkler
• Rob Painting
• Robert Way
• Peter Jacobs
• Andrew Skuce
• Ari Jokimäki
• Riccardo Reitano
• Rob Honeycutt
• Wendy Cook
• Phil Scadden
• Glenn Tamblyn
• Anne-Marie Blackburn
• John Hartz
• Steve Brown
• George Morrison
• Alexander C Coulter
• Martin B Stolpe

Then, US blogger Brandon Shollenberger made the revelation that he had managed to access the missing data online. He describes how he did it:

You see, months back I noticed referral links from the secret-secret forum in my logs. They were in the form of, sksforum.org/thread.php?t=14499&p=18772. When I’d click on them, I’d be redirected to one of my own pages.

I quickly realized the links I was seeing were redirection links for the links people were posting in the secret-secret forum. I figured if I could see the ones going to my own site, I could probably see the ones going to other sites. I was right. Changing the number after p= resulted in a different redirection link. Not only that, but each new redirection link was one number higher than the previous.

That meant I could look at every external link anyone posted on the forum by repeatedly adding +1 to that value. I goofed off a bit and found things like:

p=760	http://en.wikipedia.org/wiki/101_Uses_for_a_Dead_Cat

But I lost interest pretty quickly. After a few months passed, I regained interest and decided to make a list of all those redirection links. I thought it’d be amusing to keep track of what the Skeptical Science team was linking to in their secret-secret forum. After all, it’s pretty silly to have a secret-secret forum while making information about what you’re discussing in the forum public.

Once I had my list, I looked through it. Most of it was dull, but I kept seeing one domain pop up over and over and over. Additionally, I noticed a lot of the links to it ended with things like “thread.php?t=1954&r=8#47354.” That was reminiscent of the links from the secret-secret forum. I checked the domain out, thinking it’d be hilarious if there was a secret-secret-secret forum. There was.

I didn’t access it though. I didn’t have any login information. An amateur might try brute-force guessing, but I’m too clever a hacker to resort to such a crude approach. Instead, I grabbed a can of Red Bull, scarfed down a Twinkie and got ready to write some awesome haxor code.

By which I mean, I looked at the list I had, found this entry:

2929	http://.../tcp_results.php

And put it into my browser’s navigation bar. 

And what did he find? I complete database of the TCP data, including all the data that Cook & Co were refusing access to – unsecured. So Brandon downloaded it and started playing around, from which he was able to analyse the ratings by each rater. His analysis showed that there were inherent biases in the raters, which would have inevitably altered the final results.

Brandon challenged Cook to respond, which he did by getting the lawyers at UQ to threaten him with legal action if he released the data, and also, laughably, with legal action if he released the letter that threatened him with legal action if he released the data. So as a good upstanding scientist, he published it anyway. You can read it here (PDF) in all its glory.  Whilst there may be genuine concerns about ‘privacy’, the reality is that the raters have all been publicly identified, and only the links between the raters and their ratings remains confidential. But UQ claim all sorts of legal problems regarding IP, none of which are backed up by any evidence. So I have submitted an FoI to UQ for release of all those agreements.

In the Weekend Australian newspaper, environment editor Graham Lloyd took up the story:

THE University of Queensland has threatened legal action to stop the release of data used in a paper that establishes a 97 per cent scientific consensus on ­anthropogenic climate change.

The paper, lead authored by John Cook, has been the subject of debate over its methodology since it was published last year.

The university said yesterday it was prepared to take legal action to protect the privacy of survey participants.

Blogger Brandon Schollenberger said UQ had written to him claiming information he had received was illegally obtained and that the matter had been referred to US law enforcement authorities. If the material were published, UQ said, it would sue for breach of copyright.

The Cook paper said that among research expressing a position on anthropogenic global warming, 97.2 per cent ­endorsed the consensus.

“Our analysis indicates that the number of papers rejecting the consensus on AGW is a vanishingly small proportion of the published research,’’ said the paper published in the journal Environmental Research Letters.

UQ’s acting pro-vice-­chancellor (Research and International) Alastair McEwan said all data substantiating the paper, Quantifying the Consensus on Anthropogenic Global Warming in the Scientific Literature, had been published on Skepticalscience.com.

“UQ has therefore published all data relating to the paper that is of any scientific value to the wider community,” he said.

This was then followed up by an online statement from UQ in similar terms. UQ justified withholding the data on the grounds that it may be used to identify participants, but the rater IDs themselves could easily be anonymised – rather than using the number from the SkS user data spreadsheet, they could be relabelled 1, 2, 3 etc. Richard Tol and others would still be able to analyse the data by rater, without the identity of those raters being compromised.

More to come, no doubt.