Basically, having been set to review two new websites which purport to
help one sort reliable journalism from "spin," or "bias," or some other
bugaboo of present-day conversation about the media, I can tell you
straight up that you're still better off relying on your own common
sense and using a pinch of skepticism when reading anything. If you're
worried about bad coverage, read a reputable paper or its website, or
maybe try the BBC.
I'm not really surprised that neither turned out to be awfully impressive. Both sites, at root, rely on their users to review the news, and rely on a lot of people doing so in order to get at the kind of cumulative accuracy of sites like Wikipedia. They forget, I think, that Wikipedia took years to build, and that the process of drafting an article involves dozens, in some cases hundreds or thousands, of inaccurate revisions gradually being weeded out through vicious behind-the-scenes arguments on message boards. In the most contentious cases, especially those dealing with politics (and, indeed, news), it required the adjudication of Wikipedia's editors, long-established users with special rights to allow or forbid changes. If you check out one of these arguments, they tend to rely on citation and other forms of verification----all of which is to say, that by the time something's reached the standard which people have compared to the Encyclopedia Britannica, it's no longer news.
I'm not really surprised that neither turned out to be awfully impressive. Both sites, at root, rely on their users to review the news, and rely on a lot of people doing so in order to get at the kind of cumulative accuracy of sites like Wikipedia. They forget, I think, that Wikipedia took years to build, and that the process of drafting an article involves dozens, in some cases hundreds or thousands, of inaccurate revisions gradually being weeded out through vicious behind-the-scenes arguments on message boards. In the most contentious cases, especially those dealing with politics (and, indeed, news), it required the adjudication of Wikipedia's editors, long-established users with special rights to allow or forbid changes. If you check out one of these arguments, they tend to rely on citation and other forms of verification----all of which is to say, that by the time something's reached the standard which people have compared to the Encyclopedia Britannica, it's no longer news.
In fairness, though, I should distinguish the two--Newstrust.net seems like a genuine, and very sincere, effort, which simply hasn't yet got off the ground (and, I will argue, isn't ever likely to, not being terribly well-conceived), while Spinspotter, as Mark Liberman argued on Language Log, is at best an effort to spin a minimal product into something useful, and, much more likely, just a con. As Liberman writes, that journalists writing about Spinspotter have so far taken its claims more or less at face value is better indictment of the power of spin in the journalism than anything the service has done. I'm going to give you reviews of the two sites first, and then write a little about why I think they not only aren't but couldn't be done well; I've chosen to ghetto this at the bottom of the entry, as it is really speculation.
I'm tempted to beat up on Spinspotter first, as, well, I gotta admit I get a sick pleasure out of writing bad reviews. But that'd be a cheap trick, especially as I haven't anything better than Liberman (same link as above)--who, if you're curious about Spinspotter, you should read, as he seems to have spent much longer looking into the device than it deserves. He demonstrates quite rigorously that there isn't anything there, and has some nice work on the coverage of its release. I'll write about Spinspotter, using a lot of his work, below, but I recommend you read it instead of me.
--------------
NewsTrust is clearly genuine. Its website has content, the featured stories seem to make sense in terms of being about the matters people are probably trying to find out about today (I haven't been watching it except since ten a.m. this morning, so I can't say how well it performs at times when the top story of the day is less clear than it's been since our economy started going the way of the Weimar Republic). But I can't really see what it's for. I don't want to judge it by an imposed standard, so here's what NewsTrust has to say for itself on its About page:
Well, I can't say I've found any news literacy tools on the website, but maybe they're coming. It seems to be the feeds that are at the core of the website's mission. Well, feeds you can get a lot of places; NewsTrust seems to be claiming that its feeds are vetted for reliability. Looking at it, I can't really see enough evidence for this, nor do they seem to be especially interesting.
Let's have a look. Here's a version of the NewsTrust homepage from a little after noon on Saturday, 20 September:
So, first of all, it is pretty boring. There's only one story visible at once, about the obvious topic of the day, the economy, and it's from the Economist. So poor marks to the site as an agglomerator--as you look down the page, the links are all leading articles from the major newspapers, Salon, and leading newsmagazines like Newsweek. It did have an article from the British Independent, which is going a little outside the box for an American site, but hardly to the standard of a good online newsfeed in terms of finding otherwise little-known stories. Nor does it seem to be especially good at picking the news--the stories about the economy are all general articles, from today, about the state of the economy, and not about today's developments. This isn't awfully surprising--according to How These Stories Were Chosen, the stories were chosen by their ratings on ten "good journalism" values, and not by comparison with other sources' top stories, or indeed anyone's judgment of what people want to know on a particular day. So, it doesn't seem like the site's a very good place to go if you want to find out what the news is without having first read it somewhere else, but somewhere you go to check something you've read in another source.
Now that we've ruled out everything else, does it seem to be especially reliable? I can't see any reason to think so. Following the link to the reviews of the top story on the site, the Economist's "No End in Sight to the Financial Crisis," we find that it achieved this place after being reviewed well by three people. Looking through a number of articles, including some a few days old, I didn't find any reviewed by more than three or four people. NewsTrust doesn't consider a story to have a rating if only one or two people have looked at it, but it does post such stories in the same places as though that are rated. Proverb or no proverb, three isn't a crowd in the context of "the wisdom of crowds." If the website is going to be useful, then, we have to be willing to trust a selection process that involves significantly less people reviewing a story than used to do so on my high school newspaper. So, are these three experts, then? Not especially. None claim any qualification in economics. Jeanne Roberts, according to her NewsTrust profile, is a freelance writer in Minnesota. Derek Hawkins is a recent graduate of Northwestern who works for the website. And Jack Dinkmeyer is a "film consultant" who likes NewsTrust because it "gives me the opportunity to talk back to the news media after eight frustrating years gritting my teeth in silence about the neocon bias from the supposedly 'liberally biased media.' Keep in mind: media is plural--same goes for data; and if you're not angry, you haven't been paying attention." That I am being asked to trust this man's judgment to find me with neutral and honest reporting above that of the editorial boards of established media shocks me to an extent that I feel can be expressed only with the moronic simplicity of an emoticon. I will, therefore, now use one: :-o
-------------------
I am sympathetic to what NewsTrust is trying to do, although I don't think they've succeeded, but SpinSpotter is actually dishonest. It talks very big indeed:
Basically, it claims to have an algorithm to detect bad writing in newspapers, and, as the Liberman article linked to above quite effectively shows, it doesn't. I am willing to believe that, when its creators realised that it was impossible to build a machine that could read, which is essentially what they promised, they decided to release it anyway, hoping that enough people would be drawn to the site to build a functioning online community of people willing to criticise the news for them. If so, it was the hope of the Ponzi schemer, albeit one which will cost the public little. Alternatively, it could simply be a profoundly cynical attempt to grab a little advertising revenue with a glossy but very simple product (I saw an advertisement for Dianetics on the site earlier today, which is profoundly suspicious--though, as I can't find it now, I wouldn't claim you have any reason to believe me).
I won't bother to wade into talk about whether it is possible to find "spin" or "bias" with a machine, or whether SpinSpotter's list of reportorial crimes are really worth hunting down, nor to mock it for its incorrect definition of the passive voice (well, now I have brought it up, I might as well tell you that if you scroll down from that last link, you'll see that their example of the passive voice is in fact written in the active voice). The problem is that, as Gertrude Stein famously said of Oakland, there's no there there (there is now in Oakland a recently-redeveloped tower with a little blue flag on top bearing the word "There," which is kind of cute). It doesn't do anything.
Download the program (I recommend you don't--in fact, as soon as I finish writing this section I intend to get rid of it, as I suspect it's collecting information about my computer to sell, or doing something nasty, as it clearly isn't doing what it's supposed to), put it on its most sensitive setting, and you can trawl through pages of NYTimes.com, LATimes.com, and even Foxnews.com, without seeing once the little red icon that tells you that the device has found spin. In fact, to find some I had to go to a New York Times blog post about Spinspotter, which linked to a few pages that had results (the blog post itself had flags on it, but they were from the site's creators, defending themselves). Well, here's what you get:
So, it didn't like the headline, "How plan protects taxpayers," since the article was about the opinion of only one expert, and didn't like the subhead, "One expert gives treasury a good grade. A bond guru even says the government could make money." The subhead seemed to me to dispel the sourcing problem of the headline, but evidently you have to source "guru" because it makes him sound too competent. Personally, having grown up in L.A., a town where we have a lot of gurus, I can't say the term inspires any particular confidence in me, but maybe I'm jaded. Then the article wrote that the treasury "had to" bail out Freddie Mac and Frannie Mae, and I suppose SpinSpotter was right that they probably should have written "chose to," and then it tells us that expert is also the president of a website called governmenttakeover.org, which sounds quite interesting, but clearly didn't come from the algorithm. Seeing as this is the best the New York Times or I could find, and it is clearly the work of a human rather than the advertised machine, it isn't much.
Spinspotter does have excellent PR, though. It successfully got the attention of a number of media outlets, of which only the Times noticed that it doesn't actually do anything, and it has got a very, very glossy and glib little cartoon advertising it on YouTube. Although I've learned to embed videos since last week (and, yes, Mr. Cooper, gone back and done it where they're relevant in my last post), I refuse to embed this one, because it is, after all, spin.
----------------
I suspect that the model NewsTrust is using doesn't have all that much potential. It seems like it may simply be the Nupedia version of Digg. Everyone, these days, knows how Wikipedia came into being--it began as an effort to create a free online encyclopedia from the work of experts, with a several-stage process of peer review, and after months two articles had been finished, and less than a hundred begun. Jimmy Wales then added a freely-editable parallel site for drafts, and this grew with breathtaking speed and begat Wikipedia. NewsTrust, likewise, has established a hierarchy of the expertise of its users, promising that all its reviews will come from qualified people, and, while it's gotten a bit farther than Nupedia did, actually managing to get articles on the website, hasn't got enough volume of reviews to make them seem reliable. This isn't any great tragedy--our society already has wonderfully effective ways to harness the knowledge of professionals, which, in this case, are so-called legacy media outlets. They demand to be paid in return for their expertise, but that's what makes them professionals. The open, unpaid model which has made Wikipedia, and blogging, successful, doesn't purport to provide definitive answers, but does allow amateurs to share their thinking in new and extremely effective ways. But definitive answers remain the province of older forms.
Of course, sites like NewsTrust and Spinspotter claim that the old media aren't really definitive, that they aren't always right and need to be corrected. And, of course, this is true. But does the intensely detailed criticism they both claim to provide really have the potential to help much? If you look back at the reviews of the Economist article mentioned above, you'll see that the reviews relate purely to writing style--the article is called "insightful," or said to have enough information. The reviewers clearly have--as their mini-CVs imply--no independent knowledge of whether the article is right or wrong.
The reviews I've read on NewsTrust remind me of the controversies you see in the comment areas of blogs. Writers there often make sophisticated arguments, but inevitably they degenerate into calling out niggling inaccuracies in each other's reasoning, and then, worse, into responding by offering subtle redefinitions of their own writing--"I think you'll find that I wrote only that the rockets cannot be proved to have been fired by Islamic extremists, and not that they were not, which, being in the present tense, by no means means that it was impossible for Islamic extremists later to be proved to have fired the rockets. Furthermore, OldHickory038, you call them mortars, which clearly shows you have no idea what you're talking about." This is an unrealistic example only in that it doesn't include the words "moron" or "ignorant." These debates quickly take on the tenor of the most convoluted of the semantic debates Medieval theologians used to engage in over charges of heresy.
All of this kind of writing seems, to me, to be rooted in the fundamentally bizarre idea that it is possible to reach the truth about something by a sufficiently precise revision of what's been written about it. The truth about the economy, or the Israeli-Palestinian conflict, is of course something external to what's been written about it, and it seems to me patently obvious that you're going to get a better picture of the economy by reading two decent articles about it than one with a second article about the way it was written. Occasionally, of course, as journalism students, and I'm sure for those of us who go on into journalism as journalists, we need to read serious writing about the way we write, because we need knowledge about writing qua writing in order to do it. But even if we had a journalism that never broke any of the proposed rules these two sites claim to enforce, that always cited expert opinion and never judged, it would still miss things, and still get things wrong. As long as journalism is likely to remain a human enterprise, and imperfect, surely it's better to work on getting enough out that someone finds the errors of fact? Much the same way that Oliver Wendell Holmes wrote that the answer to hateful speech is usually more, more reasonable speech, the answer to inaccurate speech is usually more, more accurate speech, rather than endless drafts of revision without more reporting.
I'm tempted to beat up on Spinspotter first, as, well, I gotta admit I get a sick pleasure out of writing bad reviews. But that'd be a cheap trick, especially as I haven't anything better than Liberman (same link as above)--who, if you're curious about Spinspotter, you should read, as he seems to have spent much longer looking into the device than it deserves. He demonstrates quite rigorously that there isn't anything there, and has some nice work on the coverage of its release. I'll write about Spinspotter, using a lot of his work, below, but I recommend you read it instead of me.
--------------
NewsTrust is clearly genuine. Its website has content, the featured stories seem to make sense in terms of being about the matters people are probably trying to find out about today (I haven't been watching it except since ten a.m. this morning, so I can't say how well it performs at times when the top story of the day is less clear than it's been since our economy started going the way of the Weimar Republic). But I can't really see what it's for. I don't want to judge it by an imposed standard, so here's what NewsTrust has to say for itself on its About page:
NewsTrust.net helps people find good journalism online.
Our nonprofit, non-partisan project provides quality news feeds, news literacy tools and a trust network to help citizens make informed decisions about democracy.
The free NewsTrust.net website features daily feeds of quality news and opinions, which are carefully rated by our members, using our unique review tools. We rate the news based on quality, not just popularity. NewsTrust reviewers evaluate each article against core principles of journalism, such as fairness, evidence, sourcing and context.
Well, I can't say I've found any news literacy tools on the website, but maybe they're coming. It seems to be the feeds that are at the core of the website's mission. Well, feeds you can get a lot of places; NewsTrust seems to be claiming that its feeds are vetted for reliability. Looking at it, I can't really see enough evidence for this, nor do they seem to be especially interesting.
Let's have a look. Here's a version of the NewsTrust homepage from a little after noon on Saturday, 20 September:
So, first of all, it is pretty boring. There's only one story visible at once, about the obvious topic of the day, the economy, and it's from the Economist. So poor marks to the site as an agglomerator--as you look down the page, the links are all leading articles from the major newspapers, Salon, and leading newsmagazines like Newsweek. It did have an article from the British Independent, which is going a little outside the box for an American site, but hardly to the standard of a good online newsfeed in terms of finding otherwise little-known stories. Nor does it seem to be especially good at picking the news--the stories about the economy are all general articles, from today, about the state of the economy, and not about today's developments. This isn't awfully surprising--according to How These Stories Were Chosen, the stories were chosen by their ratings on ten "good journalism" values, and not by comparison with other sources' top stories, or indeed anyone's judgment of what people want to know on a particular day. So, it doesn't seem like the site's a very good place to go if you want to find out what the news is without having first read it somewhere else, but somewhere you go to check something you've read in another source.
Now that we've ruled out everything else, does it seem to be especially reliable? I can't see any reason to think so. Following the link to the reviews of the top story on the site, the Economist's "No End in Sight to the Financial Crisis," we find that it achieved this place after being reviewed well by three people. Looking through a number of articles, including some a few days old, I didn't find any reviewed by more than three or four people. NewsTrust doesn't consider a story to have a rating if only one or two people have looked at it, but it does post such stories in the same places as though that are rated. Proverb or no proverb, three isn't a crowd in the context of "the wisdom of crowds." If the website is going to be useful, then, we have to be willing to trust a selection process that involves significantly less people reviewing a story than used to do so on my high school newspaper. So, are these three experts, then? Not especially. None claim any qualification in economics. Jeanne Roberts, according to her NewsTrust profile, is a freelance writer in Minnesota. Derek Hawkins is a recent graduate of Northwestern who works for the website. And Jack Dinkmeyer is a "film consultant" who likes NewsTrust because it "gives me the opportunity to talk back to the news media after eight frustrating years gritting my teeth in silence about the neocon bias from the supposedly 'liberally biased media.' Keep in mind: media is plural--same goes for data; and if you're not angry, you haven't been paying attention." That I am being asked to trust this man's judgment to find me with neutral and honest reporting above that of the editorial boards of established media shocks me to an extent that I feel can be expressed only with the moronic simplicity of an emoticon. I will, therefore, now use one: :-o
-------------------
I am sympathetic to what NewsTrust is trying to do, although I don't think they've succeeded, but SpinSpotter is actually dishonest. It talks very big indeed:
Now there's a website and software tool that exposes news spin and bias, misuse of sources, and suspect factual support. At SpinSpotter, you'll experience the news in a profound new way. Yes, the truth is back in town. (From the front page, linked above)
Basically, it claims to have an algorithm to detect bad writing in newspapers, and, as the Liberman article linked to above quite effectively shows, it doesn't. I am willing to believe that, when its creators realised that it was impossible to build a machine that could read, which is essentially what they promised, they decided to release it anyway, hoping that enough people would be drawn to the site to build a functioning online community of people willing to criticise the news for them. If so, it was the hope of the Ponzi schemer, albeit one which will cost the public little. Alternatively, it could simply be a profoundly cynical attempt to grab a little advertising revenue with a glossy but very simple product (I saw an advertisement for Dianetics on the site earlier today, which is profoundly suspicious--though, as I can't find it now, I wouldn't claim you have any reason to believe me).
I won't bother to wade into talk about whether it is possible to find "spin" or "bias" with a machine, or whether SpinSpotter's list of reportorial crimes are really worth hunting down, nor to mock it for its incorrect definition of the passive voice (well, now I have brought it up, I might as well tell you that if you scroll down from that last link, you'll see that their example of the passive voice is in fact written in the active voice). The problem is that, as Gertrude Stein famously said of Oakland, there's no there there (there is now in Oakland a recently-redeveloped tower with a little blue flag on top bearing the word "There," which is kind of cute). It doesn't do anything.
Download the program (I recommend you don't--in fact, as soon as I finish writing this section I intend to get rid of it, as I suspect it's collecting information about my computer to sell, or doing something nasty, as it clearly isn't doing what it's supposed to), put it on its most sensitive setting, and you can trawl through pages of NYTimes.com, LATimes.com, and even Foxnews.com, without seeing once the little red icon that tells you that the device has found spin. In fact, to find some I had to go to a New York Times blog post about Spinspotter, which linked to a few pages that had results (the blog post itself had flags on it, but they were from the site's creators, defending themselves). Well, here's what you get:
So, it didn't like the headline, "How plan protects taxpayers," since the article was about the opinion of only one expert, and didn't like the subhead, "One expert gives treasury a good grade. A bond guru even says the government could make money." The subhead seemed to me to dispel the sourcing problem of the headline, but evidently you have to source "guru" because it makes him sound too competent. Personally, having grown up in L.A., a town where we have a lot of gurus, I can't say the term inspires any particular confidence in me, but maybe I'm jaded. Then the article wrote that the treasury "had to" bail out Freddie Mac and Frannie Mae, and I suppose SpinSpotter was right that they probably should have written "chose to," and then it tells us that expert is also the president of a website called governmenttakeover.org, which sounds quite interesting, but clearly didn't come from the algorithm. Seeing as this is the best the New York Times or I could find, and it is clearly the work of a human rather than the advertised machine, it isn't much.
Spinspotter does have excellent PR, though. It successfully got the attention of a number of media outlets, of which only the Times noticed that it doesn't actually do anything, and it has got a very, very glossy and glib little cartoon advertising it on YouTube. Although I've learned to embed videos since last week (and, yes, Mr. Cooper, gone back and done it where they're relevant in my last post), I refuse to embed this one, because it is, after all, spin.
----------------
I suspect that the model NewsTrust is using doesn't have all that much potential. It seems like it may simply be the Nupedia version of Digg. Everyone, these days, knows how Wikipedia came into being--it began as an effort to create a free online encyclopedia from the work of experts, with a several-stage process of peer review, and after months two articles had been finished, and less than a hundred begun. Jimmy Wales then added a freely-editable parallel site for drafts, and this grew with breathtaking speed and begat Wikipedia. NewsTrust, likewise, has established a hierarchy of the expertise of its users, promising that all its reviews will come from qualified people, and, while it's gotten a bit farther than Nupedia did, actually managing to get articles on the website, hasn't got enough volume of reviews to make them seem reliable. This isn't any great tragedy--our society already has wonderfully effective ways to harness the knowledge of professionals, which, in this case, are so-called legacy media outlets. They demand to be paid in return for their expertise, but that's what makes them professionals. The open, unpaid model which has made Wikipedia, and blogging, successful, doesn't purport to provide definitive answers, but does allow amateurs to share their thinking in new and extremely effective ways. But definitive answers remain the province of older forms.
Of course, sites like NewsTrust and Spinspotter claim that the old media aren't really definitive, that they aren't always right and need to be corrected. And, of course, this is true. But does the intensely detailed criticism they both claim to provide really have the potential to help much? If you look back at the reviews of the Economist article mentioned above, you'll see that the reviews relate purely to writing style--the article is called "insightful," or said to have enough information. The reviewers clearly have--as their mini-CVs imply--no independent knowledge of whether the article is right or wrong.
The reviews I've read on NewsTrust remind me of the controversies you see in the comment areas of blogs. Writers there often make sophisticated arguments, but inevitably they degenerate into calling out niggling inaccuracies in each other's reasoning, and then, worse, into responding by offering subtle redefinitions of their own writing--"I think you'll find that I wrote only that the rockets cannot be proved to have been fired by Islamic extremists, and not that they were not, which, being in the present tense, by no means means that it was impossible for Islamic extremists later to be proved to have fired the rockets. Furthermore, OldHickory038, you call them mortars, which clearly shows you have no idea what you're talking about." This is an unrealistic example only in that it doesn't include the words "moron" or "ignorant." These debates quickly take on the tenor of the most convoluted of the semantic debates Medieval theologians used to engage in over charges of heresy.
All of this kind of writing seems, to me, to be rooted in the fundamentally bizarre idea that it is possible to reach the truth about something by a sufficiently precise revision of what's been written about it. The truth about the economy, or the Israeli-Palestinian conflict, is of course something external to what's been written about it, and it seems to me patently obvious that you're going to get a better picture of the economy by reading two decent articles about it than one with a second article about the way it was written. Occasionally, of course, as journalism students, and I'm sure for those of us who go on into journalism as journalists, we need to read serious writing about the way we write, because we need knowledge about writing qua writing in order to do it. But even if we had a journalism that never broke any of the proposed rules these two sites claim to enforce, that always cited expert opinion and never judged, it would still miss things, and still get things wrong. As long as journalism is likely to remain a human enterprise, and imperfect, surely it's better to work on getting enough out that someone finds the errors of fact? Much the same way that Oliver Wendell Holmes wrote that the answer to hateful speech is usually more, more reasonable speech, the answer to inaccurate speech is usually more, more accurate speech, rather than endless drafts of revision without more reporting.
Leave a comment