Some thoughts on Geek Manifesto

If you ask for what you want, you might just get it, but not quite how you expect.

Geek Manifesto is a detailed summary of the “awakening of the geeks” over the last few years. Covering libel, Science is Vital, going from the #Quacklash to the Nutt Case (congrats to Jenny for the most positive name in the book), Mark gives a great recent history tour of science policy and politics.

For policy makers, the following may be true of science history:

Each time history repeats itself, the price goes up (unknown).

But, for the geeks, the following book quote is probably the critical one:

“They didn’t do this in quite the way that geeks had anticipated”

Mark makes good points, and he’s evolving the book in the talks he’s giving, but if party politics is a creature of ideology, “geeks” need to understand how to operate in that world. That the hints are at best oblique is a problem that needs to be solved for the book to reach the highly ambitious goals the community has set.

No2ID has recently learnt the hard way.

The coalition came in promising a new wave of civil liberties. 2 years in, that promise is looking a little more fleshed out than it was then, and that wasn’t quite what No2ID thought they’d won. The ID card database has gone; but No2ID are busier than ever (and less happy).

As Privacy International said at the recent Scrambling For Safety, “the names don’t matter”. The policy juggernaut rolls on, on a pretty similar trajectory to before, whoever is in charge.

The biggest omission from the book, that can be trivially corrected in a future edition, is an understanding that, what you think you want , what “they” think you ask for, and what you get, can be radically different things.

Unless someone in your community is permanently engaged, you will only find the difference between the first two when the third disappoints. That permanent engagement takes resources.

If you’ve ever heard an epidemiologist and a social scientist  talk about the same data from a different perspective (replacing disciplines as appropriate), they often fail to understand each other. There is a language barrier which often results in specialists using either different words to mean the same thing, or the same words to mean totally separate concepts.

But that gap is trivial compared to the difference between people who believe more in facts than winning and argument; and those who go the other way. This is a fundamental part of C.P. Snow’s divide, which, in this context, is science vs politics (a topic Bill Thompson & Ben Goldacre covered at OpenTech in 2009).

“Evidence Based Policy” means different things to different people, in part, depending on whether your base is evidence or politics in foundation. Fundamentally, a key differentiator between “the geeks” and others, is whether you care more about facts than being right.

Mark is right that policy evidence can get fixed; but the end of the book doesn’t seem to offer any practical recipe, idea, or guidance on doing so. That’s perfectly fine, but Manifestos generally do. It’s the doughnut, without the jam. In 5 years time, this book will be even more important than it is now.

However, to quote Clay Johnson, “it’s time to go all Lessig on the place”. Soon.

PS — to be clear, if you’re not sure about the book, I strongly suggest you go along to hear Mark speak, and ask your question. You’ll (probably) get a highly useful and insightful answer, but probably one that was missing from the book. But then, maybe the book wasn’t aimed at you.

For the next 2 years, the council will be finely balanced, with 21 Lib Dems, and 21 others (the Mayor has a casting vote which will go with the controlling group– I’m not sure if that is equivalent to Speaker Denison’s Rule).

The advantage that comes from the LibDems being a finely balanced majority is that they traditionally like ideas rather than ideology. That is most clearly true in Cambridge’s LibDem MP.

With such a fine balance of consensus, power and interested parties, people within all parties can be persuaded to look at different things that may improve democracy and engagement, not just amongst followers of Richard’s blog, but all those who care about living in Cambridge (which is different to living politics in Cambridge).

What could that look like? 

For the next year or more, the parties in Cambridge have a strong incentive not to block everything, but to try things.

“Engaging in politics” is not something everyone must do, but caring about where you live is something most people already do.

How could it be easier for them to add more to Cambridge?

After finishing some server changes late one summer night in 2006, I checked that TheGovernmentSays was still working before heading to bed late. In the few minutes before, there had been a couple of signups for email alerts, for news around airlines and liquids, from gov.uk email addresses. It was unusual, but clearly working, so I went to bed.

About an hour later, airline security protocols changed somewhat.

The Government Says was something I built when I was trying to create an RSS reader, and something else happened instead. It did rss/email alerts on arbitrary keywords from any or all departments, including limits for UK regions. If you want to know whenever any press release about Wales mentioned the word Iguana, or any government press release anywhere mentioned “liquids”, it did that. It still does, for free. With no publicity, ever, it tells 2000 people things they care about.

The source of all that data was the Central Office of Information, which collated all press releases, which I then scraped from one place. CoI has now been closed, so it doesn’t do that any more. And a significant proportion of rss feeds from government are unhelpful in various ways (broken being a description of many of them).

I’m wondering what the future is for the site.

There is clearly a need for topic based information from Government, irrespective of source; but is this something that GDS itself should do? I’m not sure.

What would a modern replacement look like? I suspect pretty similar in many ways – maybe with twitter bent added for hashtagged posting. TGS has run successfully with minimal work since 2004 – I get email alerts about new departments/agencies/inquiries/panels/courts/etc a couple of times a month, and have 95% of the work automated for that.

Maintaining this list on an ongoing basis without CoI will be “quite” hard. I include below the full list of national departments/ agencies/etc that CoI covered. That is far wider than the GDS remit; and not all have rss feeds. This list is just Central Government, there are the regional variants too.

Maybe there could be an office of information which receives all of that official stuff coming in, and then reposts it in various ways? You know, centrally. I have no idea what that group could be called however.

Your ideas on what TGS could become are welcome in the comments.

216 agencies:

• Acas News
• Adjudicators Office News
• Adult Learning Inspectorate News
• Advisory Committee on Business Appointments News
• Al-Sweady Public Inquiry News
• Allied Maritime Component Command Headquarters Northwood News
• Army Prosecuting Authority News
• Assets Recovery Agency News
• Association Of Chief Police Officers News
• Association Of Police Authorities News
• Attorney General’s Office News
• Autolink News
• Awards for All News
• Bank of England News
• Big Lottery Fund News
• Boundary Commission for England News
• British Geological Survey News
• British Geological Survey News
• British National Space Centre News
• Buckingham Palace News
• Cabinet Office News
• Care Quality Commission News
• Care Support Independence News
• Casino Advisory Panel News
• Central Arbitration Committee News
• Central Office of Information News
• Charity Commission News
• Child Exploitation And Online Protection Centre (CEOP) News
• Child Maintenance and Enforcement Commission News
• Civil Nuclear Constabulary News
• Clarence House News
• Commission for Equality and Human Rights News
• Commission for Racial Equality News
• Committee On Standards In Public Life News
• Commonwealth War Graves Commission News
• Competition Commission News
• Concerted Inter-Agency Criminal Finances Action Group/Payback News
• Consumer Council for Water News
• Consumer Direct News
• Criminal Cases Review Commission News
• Criminal Injuries Compensation Authority News
• Cycling England News
• De Menezes Inquiry News
• Defence Logistics Organisation News
• Department for Business, Enterprise and Regulatory Reform News
• Department for Business, Innovation and Skills News
• Department for Children, Schools and Families News
• Department for Communities and Local Government News
• Department for Constitutional Affairs – Judicial Appointments News
• Department for Culture, Media and Sport News
• Department for Education And Skills News
• Department for Environment, Food And Rural Affairs News
• Department for International Development
• Department for Transport News
• Department for Work and Pensions News
• Department of Energy and Climate Change News
• Department of Health News
• Department of Innovation, Universities and Skills News
• Department of Trade and Industry News
• Deputy Prime Minister’s Office News
• Directgov News
• Disability Rights Commission News
• Drinking Water Inspectorate News
• Driver and Vehicle Licensing Agency News
• Driving Standards Agency News
• England Implementation Group News
• England’s RDAs News
• Equality and Human Rights Commission News
• European Regional Development Fund News
• Export Credits Guarantee Department (ECGD) News
• Financial Services Authority News
• Firebuy News
• Food Standards Agency News
• Foreign and Commonwealth Office News
• Forestry Commission News
• Gambling Commission News
• Gangmasters Licensing Authority News
• Government Equalities Office News
• Government News Network News
• HM Courts Service News
• HM Crown Prosecution Service Inspectorate News
• HM Inspectorate of Constabulary News
• HM Land Registry News
• HM Revenue & Customs News
• Health Service Commissioner (OMBUDSMAN) News
• Health and Safety Commission News
• Health and Safety Executive News
• Highways Agency News
• Home Office News
• House of Lords Appointments Commission News
• Human Fertilisation Embryology Authority News
• Implementation Review Unit News
• Independent Case Examiner News
• Independent Chief Inspector of the UK Border Agency News
• Independent Commission on Banking News
• Independent Complaints Reviewers Office News
• Independent Police Complaints Commission News
• Independent Safeguarding Authority News
• Insolvency Service News
• Intelligence and Security Committee News
• Judicial Communications Office for England & Wales News
• Knowledge to Innovate News
• Knowledge to Innovate News
• Lantra News
• Learning And Skills Council News
• Learning And Skills Council News
• Leasehold Advisory Service News
• Local Better Regulation Office News
• London Youth Crime Prevention Board News
• Lord Goldsmith QC Citizenship Review News
• Low Pay Commission News
• Manchester Aiport News
• Marine Fisheries Agency News
• Marine Management Organisation News
• Maritime And Coastguard Agency News
• Met Office News
• Metropolitan Police Authority News
• Midlands World Trade Forum News
• Ministry of Defence News
• Ministry of Justice – Judicial Appointments News
• Ministry of Justice News
• NDS2 News
• NHS Expert Patients Programme News
• NHS Heart Improvement Programme News
• NHS Modernisation Agency News
• NS&I News
• National Audit Office News
• National Business Travel Network News
• National College For School Leadership News
• National Crime Squad News
• National Statistics News
• News Coordination Centre News
• News Distribution Service News
• News from Department of Constitutional Affairs
• News from English Heritage
• Norfolk Constabulary News
• OFWAT News
• OGCbuying.solutions News
• Office For Criminal Justice Reform News
• Office For Standards In Education (OFSTED) News
• Office of Fair Trading News
• Office of Government Commerce News
• Office of Rail Regulation News
• Office of The Schools Adjudicator News
• Office of the Children’s Commissioner News
• Office of the Civil Service Commissioners News
• Office of the Commissioner for Public Appointments News
• Office of the Deputy Prime Minister News
• Office of the Traffic Commissioner News
• Parliamentary Commissioner For Administration (OMBUDSMAN) News
• Parole Board For England And Wales News
• Pensions Commission News
• Personal Account Delivery Authority News
• Polar Quest News
• Postcomm – the Postal Services Commission News
• Prime Minister’s Office News
• Public Health Observatories of England News
• Quality Improvement Agency News
• Queen's Awards for Voluntary Service News
• Redfern Inquiry into Human Tissue Sampling in connection with UK Nuclear Facilities News
• Regional Minister For London News
• Regional Minister For The East Midlands News
• Regional Minister For The East Of England News
• Regional Minister For The North East Of England News
• Regional Minister For The North West News
• Regional Minister For The South West News
• Regional Minister For The West Midlands News
• Regional Minister For Yorkshire And The Humber News
• Regional Minister for the South East News
• Renewable Fuels Agency News
• Residential Property Tribunal Service News
• Residential Property Tribunal Service News
• Respect Programme News
• Revenue and Customs Prosecution Office News
• Royal Court of Jersey News
• Rural Development Service News
• Rural Payments Agency News
• Security Industry Authority News
• Serious Organised Crime Agency (SOCA) News
• Skills Development Scotland News
• Skills Funding Agency News
• South West Grid for Learning News
• St James’ Palace News
• State Veterinary Service News
• Strategic Advisory Board for Intellectual Property Policy News
• Strategic Rail Authority News
• Student Loans Company News
• Tate Britain News
• Tate News
• Technology Strategy Board News
• The Baha Mousa Public Inquiry News
• The Court of Appeal, Jersey News
• The Identity and Passport Service News
• The National Archives News
• The Office of Rail Regulation News
• The Pensions Regulator News
• The Queen’s Awards for Enterprise News
• The Review Of The Funding Of Political Parties News
• The Scottish Parliament News
• Tower Hamlets News
• Tower Hamlets News
• Training and Development Agency for Schools News
• Treasury News
• Tribunals Service News
• UK Border Agency News
• UK Film Council News
• UK Intellectual Property Office News
• UK Ship Register News
• UK Statistics Authority News
• UK Trade & Investment News
• United Nations – Habitat News
• University For Industry News
• Valuation Office Agency News
• Water Services Regulation Authority News

Flack is a organisation actively doing great work in and for the community of Cambridge. It’s a social enterprise of, by and for those with experience of homelessness, but it has the monthly task of producing a magazine and other material that sells well and provides commercial income. That drumbeat of the relentless plodding of time means there’s limited time for things that probably won’t help. Handling data, especially open data is like having a scene of rolling hills and fields, full of haystacks, in a couple of which there might be some needles. My role was the equivalent of showing up (and leaving them with) with some field-scale metal detectors. There’s still “straw” to go through, but it gives direction and confidence that there’s something useful in there somewhere

One of the things about Facebook and Twitter, is that people can say something that “others” think stupid, and the horse has almost immediately bolted. Previously, your mates would laugh and forget about it, and no one else would know, unless you were “important” enough for anyone else to care.

There was an incident involving Chris Brown at the Grammys, and his history of domestic violence. I have no idea who Chris Brown is, why he was at a music show, but for this, none of that is relevant. What matters is that people had the ability to comment. And they did with the diverse voices and wisdom, and lack thereof, that people have. Some people commented on twitter/facebook that given the history/talent, they wouldn’t mind that being done to them. Previously, that’d disappear into looks from friends.

While those statements may be unwise, this buzzfeed piece started doing the rounds where they’d quoted some people. The piece wasn’t universally welcomed. Without comment on the content of the tweets, it was very interesting to see what parts the article itself included.

Out of 25 comments, 4 were from facebook with profile photos and only first names shown (ie surnames redacted), and 21 were from twitter with no information redacted – names and usernames were republished alongside the comment. Unless you knew someone from the firstname/picture, then the facebook quotes were anonymous; but the tweets, you could comment to people easily

That was clearly a choice, but since various media furores, I’ve been wondering about the effects of naming and new internet norms, and this formed a nice natural experiment.

A week after the piece was published, I went back to the article, and looked at the twitter accounts again, to see what had changed.

Some respect is due to the 5 accounts where nothing had changed at all – they just kept on tweeting. One more was now locked, another wiped but with a single tweet remaining, standing by what was been said, and in one case, the account details had been blacked out in the original article in the interim. The majority of the accounts, 18 of 25, had been deleted (with some now recreated by different people).

In short, this may be another measurable datapoint in the seeming evolution of names, and the chasing of people round media. That BBC piece talks about how names have evolved. In time, it’s likely that 26 September 2006 at around 9am, will become recognised as the inflection point. Technology has power, and unintended consequences on others.

I post this today as this issue seems to disproportionately affect women, and it’s International Women’s Day. Friends’ experiences and outcomes (at varying levels of severity) pretty much matches the above percentages on outcomes. It’s not a trivial issue; but it may go away over time. Unfortunately, a number of people will take a bruising to get there. 3 names might work too.

But I wonder whether this would have had the references to hit 100 million views a few years ago:

But, more usefully for this audience, the views breakdown is rather interesting: 20% of views are via mobile.

The SOCITM studies usually cited are a cherry picking of those who care about the answers because they look good, based on the selective users who do reply.

But we know something else about Local Government websites: they’re supposed to do things, not talk about things, and so we can look at service not fluff. Local DirectGov is a database of services, and is supposed to be right…

Over 5% of Local DirectGov links are clearly broken.
Over 6% are links to the same content as something in a different category. In total, ~10% of links are identifiably broken.

So here’s a breakdown of which Local Authorities might care least:

 top London Borough of Brent (some false reports, but lots of broken stuff)
     Borough of Poole
     St Helens Metropolitan Borough Council
     Wiltshire Council (Unitary)
     Solihull Metropolitan Borough Council
     Torbay Council

(But see below for why this is a undercount).

Methodology

When we turned off DirectionlessGov.com, while we knew that @GovUK would be better for national services; there was the big question about Local. DirectGov was of variable quality, there was much less variation in Local DirectGov.

Directionless knew what a postcode was, and if you included one in your search, it did the right thing. Directgov, did something else.

Various incidents and processes kept this issue from being addressed; and like many of the DirectGov problems, it festered on multiple levels which made solving any a pain.

And while this data has been released before (I first FoIed it about 5 years ago, and got a copy last year), what has changed?

When I first requested the data, I was told that what I was asking for didn’t exist. It did, but style of engagement continued which didn’t aid progress.

Then GovUK showed up.

There are lots of criticisms that could be levelled at the data posted, but fundamentally, a substantial chunk of it is wrong. The file originally published by Local DirectGov , the file that comes out of their systems for others to use, included characters that were not only not UTF-8, they weren’t UTF-anything. Odd characters (Windows Pound signs) are understandable in titles, random junk, not so much.

That junk was in links.

Fundamentally, those links could not ever have worked. And at no point did anyone care enough to point this out. I’m sure the people submitting that data didn’t intentionally think “I’m going to put crap in this box” (simply because those characters required yogic-guru-class key combinations), it’s just no one told them it had happened.

The Local DirectGov project has had issues, some of which aren’t at all easy to fix, but the fact that their infrastructure doesn’t check “does this link work” has directly led to a large number of those problems being visible to users. I appreciate that they’re trying to get a bureaucracy to do a thing that’s really hard for a bureaucracy, but that they were using bad tools isn’t an excuse — but it is fixable.

And so we’ve now got a 100,000 line spreadsheet of links, and they don’t know where to start. So I wrote some code (it’s on github).

The output of data is here. It currently only does the advanced checks against a random subset of files for duplicate content (since 404 pages may lie about everything, but 404 pages within the same authority are generally pretty similar). The full check will take about 4 days to run (it could be parallelised). The script as is currently takes about 2 hours.

@GovUK: If you get council to fix 5% of these 10,000 broken links, you’ll start on the way to having a better service for users, by not dropping them onto a page which certainly doesn’t work. How many of the remaining 90,000 pages are also wrong, is a somewhat different problem…

I have a few ideas.

But I also know, that the next time SOCITM publish a report on council websites, that there’s code available to look at how irrelevant their figures are in practice, and it can be done in 2 hours…

The above was done on or around the 14th Feb 2012 – the link state now will be different (hopefully better, probably worse).

Invitation to help Government shape its approach to Open Data

As part of establishing the Public Data Group (PDG) and the Data Strategy Board (DSB) – delivering on a commitment to create a Public Data Corporation – Cabinet Office proposes to set up an Open Data User Group (ODUG). This group will act as an intelligent customer on behalf of Government on Open Data, and through its chair provide advice for the DSB on where to spend funding allocated for Open Data. Building on the existing knowledge and experience among Open Data activists, users and re-users of Government data, we are seeking to develop a strong framework for the ODUG.

It is envisaged that part of this group’s influence will would stem from the production of evidence of economic or social growth; the creation of new jobs; and the emergence of a new data markets. The group will be supported by a secretariat in Cabinet Office Transparency Team.

This session is set up to allow for a frank discussion – under Chatham House rules and with the express understanding that wider communication (blogs, tweets etc) would not be appropriate – with a representative selection from the Open Data community, as well as commercial users and re-users of Open Data.

The aim of the session will be to gather views on how the ODUG needs to be developed in order to give the most effective and balanced advice for how to prioritise and spend funding allocated for Open Data. In achieving this aim, the session will focus on questions like:

• What is the appropriate formation of the ODUG?
• How can we ensure the collection and analysis of credible evidence to support Open Data purchasing decisions?
• What is the most effective way to involve as wide a representation from Open Data users and re-users as possible?

To support the aim of the session, the following agenda has been drawn up:

• Welcome and introductions
• Opening remarks from Cabinet Office – what is the current situation?
• Brainstorm Consolidation of opinion
• Closing remarks and next steps
• AOB

Especially since the diverse set of conversations I’ve heard about after that meeting are that the questions were set up to get broad answers, which were then cherry picked (sorry, “consolidated”) by those involved based on whatever it was they wanted to hear (e.g. section 6).

Anyone who was at the meeting care to comment quietly? (I wasn’t, and nor would I have expected to be). Or got a copy of the full attendee list?

I get that the reforms seem like a bad idea; lots of highly informed people I respect say so, but that doesn’t really help me understand why. I appreciate it’s not as simple as that, but there will hopefully be a simple-ish way to explain.

Ideally, something I can forward to friends who don’t give a crap about politics (I don’t have many of them, but think of it as something that your siblings might forward to their friends)..

Finally.