Newspaper Club sneaks into alpha - this is going to be amazing

One of the most potentially amazingly powerful projects from 4iP has sneaked out a bit - Newspaper club (Disclosure: they sponsored opentech earlier this year, but NC wasn't discussed as far as I'm aware).

One of the projects locally that i've started doing a bit of work with is a Manchester volunteer newspaper (the MULE). They look at a variety of things that aren't getting covered in other media - in terms of local newspapers, the only thing really left focussing inside the M60 is the Manchester Evening News), and that's extremely cosy with the council.
They'd never think of looking at something like this: Closed doors at the council.

The main issue with printing anything is that the first copies are very expensive, and soon after that the cost can come down to roughly the cost of the paper it's printed on. But doing 2 editions of 5000 copies is nearly double the cost of 1 edition of 10,000.

Now, in the MULE's single paper edition (latest PDF here that story is buried on page 5, but what would happen if MULE could upload a custom PDF with that story on page 1, and get a small run printed cheaply, and then distribute that version at the door to the town hall on the evening of a council meeting. What impact would that have on the council and it's level of belief in democracy and transparency (and quite frankly, there's only one way that it can easily go). And if that doesn't work, what happens if it gets handed out on the street in the contested wards in the week before an election.

Or what happens if it is possible to have the story about Beetham Tower using illegal wood on the front page of the version that went in every post box in that building, and the (numerous) other big buildings owned by the same company? Both those that have been built (the people who pay rent), and those that are under planning discussions (when small spanners can jam up works pretty easily). it only takes one person to have read the article and raise it, to start a process of change.

Newspapers became popular for a reason; and large parts of that reason haven't changed. The business models have; the methods of distribution have. But that's an opportunity, as well as a threat.

There are many local papers like MULE out there, the internet has let them publish online for a long time, but the big problem with hyperlocal is getting people to find out about it. Newspapers have been solving that problem for a long time by putting it through people's front doors on a regular basis with something worth reading. Mixing the two will have dramatic impact.

Online is of course vital, but offline can get people there, and get people to make things better.

Watch the students

My friend Alasdair does a huge amount of stuff, all of it well, but one of the best pieces of advice he ever gave was "watch the students".


His argument being that students have a relatively unique combination of things - lots of time, access to resources, and little money. These, when also spending large amounts time around friends with similar interests (and alcohol, which probably matters quite a lot) can create a very fluid and rapidly innovative environment, without the restriction of thinking that something's not possible or feasible, but simply going out and doing what everyone else said was impossible. But this is not new (#6).


The balance of time and money being that way round, means that much is made of little resources, and things are reused in novel and interesting ways to get things done (Emily's "fairy's well" to the right is made from a recovered large cable drum and few space pieces of wood). Other people make elephants and paint themselves blue but it might astound you just how cheaply things can happen, and just how many organisations will lend you incredible things if it's to be part of a student event.

There are few other similar environments where such ideas can be germinated, fermented and then carried out with ease.
The usual case of this can lead to some absolutely fantastic experiences.

But every silver lining has a dark cloud when we apply it to our usual themes of technology and democracy. Taking a look at London Met Students' Union, which has had to abandon their elections after their "online elections" was subject to widespread fraud (outcome, details)). Details are scarce, but it appears that someone voted many many times using other people's online voting credentials (no, they're not secure). While they traced many of the votes back to a small number of IP addresses (because the abusers haven't heard of tor?) they had no way of telling who was preventing from voting as someone had already voted as them. The entire election had to be abandoned (and will be rerun near Christmas - leaving students without elected representation for 3 months). Essex had the same problem around the same time which lead to the result changing after it had been announced (almost deja vu — paragraph 8 — re: scotland).

While, in the grand scheme of things, student union elections don't matter that much (unless you're a student in that institution needing representation - which can be vital to an individual student's degree), the fact that they can be thrown quickly, repeatedly and with devastating results is a problem whatever the level. It's also where the politicians start their politiking; and if they don't care that "online" elections can be thrown at that level, what makes you think they would they care at higher levels.

Being able to spend time to develop and implement new and novel ideas is an extremely valuable thing. But to get a silver lining, you need a dark cloud.

Some meandering thoughts on "The Vision Thing"

Amongst friends of a friend, there's a conversation going on about "the vision thing" and how their (IT) industry has lost it's way: original post, a reply, roundup.

Paul comments that most of the pitches he hears are derivative - but 95% of everything is crap, and you find a good pitch, generally you don't have time to listen to any more. More so when you consider that there are so few new ideas, and most of them come from connecting across multiple disciplines, which very few people do.

Useful comes in phases. We need time to figure out how to use tools in new ways once they're built, and also have a idea to use them for. YouTube has been around for 3 years now, but we're still using it like "TV on the internet". Granted, anyone can have a TV channel and you can embed it in web pages, but it's still pretty much the same thing. YouTube has revolutionised media distribution, but it's not revolutionised creation, and has probably done less than iMovie. The operative word being yet. All the bits that make the gmail interface be gmail existed for a number of years before gmail itself actually existed - what changed was simply someone putting them together in that way.


That's not sexy or even necessarily new, And I'm pretty sure that the pitch for gmail would have sounded really dull. Until you actually used it. This site isn't sexy, and probably wouldn't have even taken that long to build http://omnisio.com/ - but think about the implications of being able to attach arbitrary points of video together. And you can go from http://www.youtube.com/watch?v=iMyuy7yDdG4 and end up near http://www.youtube.com/watch?v=1yq0tMYPDJQ
.


When anyone can shoot a video on a £50 camcorder, that's a good reason for people who are capable of doing more being given the skills and tools to go further and use their talents, rather than be restrained by basic tools now. Over time, it will get opened to more as the barriers to entry are lowered. It's easy to claim that anyone can do video, but not everyone can do it well (cf some youtube content). And the tools aren't there yet on how to do that online, but give creative people a copy of final cut pro and magic can happen. They'll bring the good ideas themselves, and overcome all sorts of hurdles to make it happen. Good ideas have a habit of fighting to get out (and sometimes they keep score).

There's nowhere that yet integrates video seamlessly, rather than being a flashy site that says "we do video". The fact that the definitional video site is youtube, rather than say, news.bbc.co.uk (or something that just uses youtube as a platform) says how far there is to go in something that could be considered good (although I'm aware the BBC are doing a redesign soon which might make this wrong overnight).

Think of the course that "DTP" took from the mid-80s to about 2000 when amateurs stopped having to care as the tools were good enough (and the professionals could get on with the interesting stuff rather than laying out a newsletter for cash). This progression takes time, and in the words of Rosabeth Moss Kanter, "everything looks like a failure in the middle"


In a post, Paul (who started this off) wrote:

Children who can't afford shoes in Africa are given laptops so that they too can learn skills needed to write on a 'Super FunWall' and define the solidity of their friendships as nothing more than a button click

Working on a project with some friends, one person gave their reason for spending time on it as "because I want kids with a $100 laptop to find out why we're bombing their country".

If you want to see where you time is going (and you're anything like me who does most stuff by email), look at the breakdown of email in your sent-mail folder. One good definition of who you are is what you spend your time doing. And there's only one person who controls that (stolen from Tim Ferris, in a reference I can't currently find).

When is St Petersberg (not) the same as Leningrad?

If you've not already read it, see Tom Longley's first post about this before not having a clue what's below. Although I'd strongly recommend that you don't read the linked report unless you're very interested in specifics - it's really not pleasant reading.

Tom asked some people to take a look at the documents, and see what we could think of doing to fix it up into more useful ways.

The structure of each document is relatively simple, and the formatting mostly fine (three different structures, which can be identified easily with regular expressions). In fact, the whole script handling most of the 5 years (a couple of reports are very different for various reasons) is only 122 lines of code (roughly half parsing and half spreadsheet generation).

The reason for using a spreadsheet is down to simplicity - it can be done fast, and the fact that the data is only semi-structured. Plus, since it's in Excel, it can be edited in almost anything, on any platform, without a requirement for anything above windows 95. Future stuff can be based either in VB/macros or in a more intelligent spreadsheet, or in something else. But in order to operate in a country with some security concerns which are far from abstract, simplicity is useful.


TomL touched on the issue of geography. Detailed geography is useful - lat/long/date being ideal as the basic item as that is constant over time, and can be rolled into the geography as needed (no, it's not that easy, but the other options are far harder). The main problem for the existing data, at the moment, is tying together the list of names which exist over time, and putting them together in some way which we can look at from a higher level (e.g. region).

This stuff is fundamentally a qualitative time series dataset on which we then build a (somewhat depressing) quantitative time series dataset. Discontinuities in geography is not a rare or unknown problem for such conversions. But we need to know the names of places before we can start doing things with them. Town names don't change much and often it doesn't matter when they do - even if there's a formal edict changing the name to Leningrad, people will still call it St. Petersberg.

While for some purposes (constituency stuff), knowing you live in Manchester or Harare is nowhere near enough in the abstract; at the current level, we don't have better data for the past stuff (or, at the moment, any decent data), so we'll have to live without it. But it does mean that as things move forwards, we can retrofit future (hopefully good) data onto the historic data: want 2008-2010 data? then you can get your own decent geography; want 2003-2010, you can get what we can give you. It's a very common problem, and there is no single solution. But something is better than nothing, and, in this case, perfect could be the enemy of good. For legal or audit purposes, being on the right street matters; for other things, it's arguably less so.




At a mySociety meeting before Christmas, Tom Steinberg spoke with his usual eloquence about how those in the room (the leading e-democracy activists from the UK and USA) were extremely fortunate to operate in countries where we can do what we do without concern for ourselves or that of friends and family. Unfortunately, this is all to rare in the world. Democracy is a great gift from the past, and we don't own it. We're only custodians for the future; and we should believe in doing the right thing for the right reasons, and in so doing, leave things slightly better than we found them.

It's easy for someone to claim that they're doing something in the name of democracy; that should not be confused with actually doing so, especially when they're running for something.

Meinedata, mellenrummet, and iQuango.org

Meinedata is a new gift from some of the mySociety volunteers. It's a flash based tool for graphing data, which can be provided to it in the form of a spreadsheet (soon) or xml (now).

Very heavily inspired by and based on the Gapminder tools, it brings some of that style of data visualisation to abitrary data. Using the tool is the new MPtables.com which takes data about UK MPs and lets you compare them.

I've also loaded the UN Common Database in as part of iQuango.org (with the name of mellanrummet). As more and more data becomes available, there should be a tool for organisations to use to do comparisons on data, and allow their website visitors to do comparisons of data, that they're interested in. This lets those with data easily allow other people to visualise that data and compare it with other data they provide; and everyone else to find a bit more about the world, and have a better understanding than the chimpanzees.

For more technical details, see the post which follows this one.

For all the similarities with Gapminder, we apologise for any regressions and very gratefully acknowledge the vast amount of inspiration, ideas, design and motivation from Gapminder, Hans Rosling (his blog), and, certainly as importantly, TED conference for sharing their "inspired" talks online, and showing what's possible with one person's vision...

What's next?

Directionless Gov Games

Government makes available large amounts of numbers about other levels of (local) Government. It locks them up in boring spreadsheets, with dull tables, in much the same way it buries useful information behind the clunky direct.gov.uk portal.

Some of this data is dull, some of it is quite interesting. And when it's all made available in machine readable formats, it becomes easier to fiddle with it and do nice things which it wasn't actually ever intended to be used for - such as be the input to a card game. The aim is to win all the cards, by selecting a better metric than your opponent (the computer). Thanks to Matthew for the code

Why do this? Because we can (and I needed an example for the NeSS API although didn't end up using it due the lack of a postcode lookup file for each LA) and because it's useful to learn about the country as it is, rather than how we think it is (or we thought it was). There's much more than can be done in this area.

For that reason, I've also done one based on country information (see the iQuango announcement post (which escaped when I pressed the wrong button, so is incomplete - sorry)).

Why?

If you read this blog, you need to watch this 15min presentation about gapminder and data.

Talking to William a few weeks ago, he asked why I do what I do.

Not really having a good response, I blathered a bit, talking about basing decisions and processes on data not spin/ideology, and various other things which made some bit of sense, but didn't really hang together as a whole.

Hopefully, at some point, there will be a better answer, but for now, watch the presentation, as some of the reasons are shared; including the chimpanzees.