Change the Debate:

How Gaffes, Games, & Gotchas Dominated
the 2008 Presidential Primary Debates

EXECUTIVE SUMMARY

While much attention was given to the trivial questions asked in the final debate of the 2008 primary season, the problems with the primary debates went much deeper and highlight some of the broader weaknesses in the news media's coverage of the campaign. Media Matters Action Network undertook this analysis to identify the weaknesses in the media's performance during these debates in the hope that future debates will be more substantive and useful to voters.

Simply put, the primary debates were a disaster. With a few exceptions, the media figures who moderated the debates focused on endless rehashes of campaign gaffes, pointless dissections of political tactics, and personal issues that had little or nothing to do with the challenges the next president will face. Many of the substantive questions that were asked, furthermore, were of the "Let's you and him fight" variety, attempting to initiate squabbling between candidates instead of a meaningful exploration of issues.

As a result, critical issues were pushed aside. Only 9 percent of the questions addressed the economy, which has become the most important issue in the general election. Only six questions out of the total of 2,304 touched on the growing crisis in the mortgage industry, which was already making headlines in 2007. Only three questions mentioned the minimum wage, and only two questions touched on the issue of declining wages.

Other critical issues were likewise given short shrift. There were 61 questions about abortion - half as many as all other health care questions combined. There were dozens of questions about oil prices, but only three questions about conservation and renewable energy. There was not a single question about the administration's unprecedented use of signing statements, its dramatic claims of executive privilege, or its extraordinary secrecy. These 31 debates featured only one question about warrantless wiretapping and only two questions about the prison at Guantánamo.

Republicans and Democrats were equally subject to the parade of silliness. When not being forced to name their favorite Bible verse or take sides between the Red Sox and the Yankees, candidates were asked to comment on polls or political strategy. While some candidates got more substantive questions than others, almost all the candidates who were near the top of the polls found themselves answering mostly trivial questions.

Debates are the best opportunity the public has to get an extended look at the candidates, in their own words. While the candidates are ordinarily filtered through news programs that typically reduce their speeches to six-second sound bites, a debate actually allows them to answer questions at length and exchange ideas with each other. But the degree to which debates do this is dependent on those who moderate them. Those moderators set the rules, ask the questions, and guide the discussion. If they are obsessed with trivia, the debates will be obsessed with trivia, and the public will be not be led anywhere closer to an informed decision. Unfortunately, that is just what happened in the 2008 primary debates.

KEY FINDINGS

INTRODUCTION

After Sens. Hillary Rodham Clinton (D-NY) and Barack Obama (D-IL) participated in their final primary debate on April 16, the reviews were scathing. But it was not the candidates who were the target of these critiques. Instead, it was the moderators, Charles Gibson and George Stephanopoulos of ABC News, who were criticized. Tom Shales of the Washington Post wrote that the two "turned in shoddy, despicable performances." Greg Mitchell of Editor & Publisher magazine called it "perhaps the most embarrassing performance by the media in a major presidential debate in years." Many critics focused on the fact that the debate was nearly half over before a single question on a policy issue was asked; instead, the moderators focused on rehashing old gaffes and absurd questions about patriotism.

Though the ABC News debate was in many ways the worst in memory, it was actually the last step in a progression throughout the primary season. While the initial round of debates was reasonably substantive, as time went on and more debates took place, the moderators became more and more focused on political maneuvering and outright trivia, and less concerned with actual policy issues.

Media Matters Action Network analyzed all the debates broadcast on national television to chart this progression. We categorized the questions as either substantive or non-substantive, with further subcategories for the non-substantive questions. A third category -- substantive but flawed -- was used for questions that had a substantive basis but that forced candidates to respond to inflammatory hypothetical situations or to attack each other.

A variety of types of questions fell into the non-substantive category. These included "gotcha" questions focusing on prior controversial statements; questions in which a candidate was forced to defend someone else, such as a supporter; questions about a candidate's political motives for some statement or position; questions revolving around poll results, electability, and so on; and questions about personal matters not directly related to policy or the duties of the president.

Question Type

At one time or another, candidates on both the Republican and Democratic sides took issue with the questions that were being asked. Overall, there was not much difference in the way Republicans and Democrats were treated: 57 percent of the questions asked of Democrats were substantive, while 60 percent of the questions asked of Republicans were substantive.

The overall picture masks the fact that the questions asked became less substantive over time. We divided the debate season into four periods: the first half of 2007, which featured six debates; the second half of 2007, which featured 13 debates; the month of January 2008, which included the beginning of actual voting and featured nine debates; and finally, the period from February 2008 on, which featured three debates between Obama and Clinton (Sen. John McCain (AZ) had secured the Republican nomination by that point). During the first period, 66 percent of the questions were substantive, while only 23 percent were non-substantive, meaning that there were three substantive questions for every non-substantive question. But as time went on, the debates moved steadily away from policy and toward discussions of campaign maneuvering, remarks made by the candidates' supporters, personality issues, and other types of trivia. By the last debate period, the proportion of non-substantive questions (52 percent) had exceeded the proportion of substantive questions (39 percent).

ISSUES

Given the large proportion of the questions that were not substantive, it should be little surprise that 21 percent of the questions touched on no policy issue at all.1 The most common topic of discussion by far was foreign policy and national security, which included the Iraq war. No other issue area was the subject of more than 10 percent of the questions.

Questions by Topic

This is particularly notable with regard to the economy, which has become the most important issue in the fall campaign. In the primary debates, it was almost an afterthought, accounting for a mere 9 percent of questions. Moderators spent much of the time they did devote to the economy attempting to pin down candidates on whether they would use the word "recession" to describe the country's situation, or asking about the large incomes of hedge fund managers. Brit Hume of Fox asked how candidates would keep the economy going if terrorists attacked our shopping malls. But only six questions out of the total of 2,304 (less than three-tenths of one percent) touched on the growing crisis in the mortgage industry, which was already making headlines in 2007. Only three questions mentioned the minimum wage – one of which was when Chris Dodd was asked if he would work for minimum wage as president. Only two questions touched on the issue of declining wages. In other words, when they weren't ignoring the economy completely, the moderators missed the aspects of the issue that are of greatest concern to many Americans.

Other critical issues were likewise given short shrift. There were 61 questions about abortion - half as many as all other health care questions combined. There were dozens of questions about oil prices, but only three questions about conservation and renewable energy (one of which was when the candidates were asked if they themselves were using compact fluorescent bulbs). Although candidates were asked whether Dick Cheney had too much authority in the Bush administration, there was not a single question about the administration's unprecedented use of signing statements, its dramatic claims of executive privilege, or its extraordinary secrecy. These 31 debates featured only one question about wiretapping and only two questions about the prison at Guantánamo.

Public Priorities

How did the priorities of the news organizations conducting the debates compare with the priorities of the American public? To find out, we can compare the attention given various issues in the debates to the priorities expressed by the public over the course of this period. The debates took place between April 2007 and April 2008. In its polls, Gallup asks Americans what they consider "the most important problem facing this country today." We took the average of three of these Gallup polls, from April 2007, October 2007, and April 2008 as a point of comparison.

While one could reasonably argue that debates shouldn't necessarily be a perfect reflection of the public agenda, we can say nonetheless that the public had somewhat different priorities than the debate moderators.2 This was particularly true of the economy, which was the basis for only 9 percent of debate questions, but was cited as the most important issue by substantial proportions of the public, particularly in 2008. In Gallup's April 2008 poll, 56 percent of respondents cited an economic issue as the country's most important problem.

In short, when the debate moderators weren't focusing on personal and trivial questions, their priorities were far out of line with what the public found important, and those issues that have come to dominate the general election.

UNEQUAL TREATMENT

Though there was not much difference between the treatment of Democrats and Republicans overall, all candidates were not treated equally. To a great degree, it appears that the better a candidate was faring in the polls and the greater the chance that that candidate might actually win the nomination, the less likely he or she was to be asked substantive questions. It is particularly troubling that the candidates with less of a perceived chance to win could expect to be asked substantive policy questions, while the main contenders were more likely to be asked to respond to trivia.

Question by Campaign Period

The candidates who were asked the most substantive questions -- Chris Dodd, Fred Thompson, Dennis Kucinich, and others -- were not front-runners. If we look only at each party's strongest candidates (McCain, Mitt Romney, Mike Huckabee and Rudy Giuliani for the Republicans; Obama, Clinton, and John Edwards for the Democrats), we see that as a group, they ranged from a low of Romney's 51 percent and Clinton's 50 percent substantive to a high of Giuliani's 60 percent substantive, with an average of 54 percent. Among the weaker candidates, almost all did better than the leading candidates, and as a group, they averaged 65 percent substantive questions.

Question by Candidate

Moreover, while McCain was asked a higher proportion of substantive questions overall than Obama or Clinton, the questions the two Democrats were asked became steadily less substantive over time, while McCain was actually asked more substantive questions during the second half of 2007 than he had been during the first half of that year. This may or may not have something to do with the fact that during the second half of 2007, he was essentially written off as having little chance to win; once he climbed in the polls, the questions he was asked became less substantive.

Proportion of Substantive Questions

THE NETWORKS: THE GOOD, THE BAD, AND THE UGLY

Though there is plenty of legitimate criticism to be made of this election's primary debates, it would be a mistake to assume that all the networks were equally culpable. In fact, there were dramatic differences between the networks that hosted these debates. PBS hosted one nationally televised debate that we analyzed3, and every single question asked of the Democratic candidates in that debate was substantive.

Substantive Questions by Network

The second-most substantive network was Univision, which was the sole host of two debates and the co-host, with CNN, of one debate.4 At 82 percent, Univision's proportion of substantive questions was far above that of the English-language commercial networks.

If PBS and Univision receive good grades for their debate performances, then poor grades go to CNN and MSNBC/NBC, each of which hosted nine debates. Fewer than two-thirds of the questions on each of those networks' debates were substantive. Finally, Fox and ABC receive failing grades; fewer than half of the questions on each of those networks' debates were substantive.

Even within networks, some moderators' questions were more substantive than others'. Though MSNBC's numbers overall were poor, Chris Matthews scored better than all but two moderators; almost three of every four questions he asked were substantive. At the other end of the scale were Chris Wallace of Fox and George Stephanopoulos of ABC, the hosts of those two networks' prestigious Sunday-morning talk shows. Thirty-nine percent of the questions Stephanopoulos asked were substantive, while only 33 percent of Wallace's questions were substantive.5

Substantive Questions by Moderator

The two moderators who asked the most substantive questions were both from Univision, the Spanish-language network. Univision sponsored one Republican and one Democratic debate, both of which were moderated by Maria Elena Salinas and Jorge Ramos; Ramos also moderated a joint Univision/CNN debate. Eighty-eight percent of the questions Salinas asked were substantive, as were 73 percent of Ramos' questions.

CONCLUSION

Debates can be extraordinarily valuable for citizens looking to assess the men and women who would be president. Unlike reports about the campaign on the evening news, where everything a candidate says in a day is reduced to a six-second sound bite, in a debate, the public has the opportunity to hear candidates speak at length. We can learn about their ideas, their proposals, and their interpretation of current events. That is, we can do so if those responsible for bringing the debates to us -- the television networks -- moderate the debates in such a manner as to make substantive discussion possible.

But when they focus on trivia, attempt to get candidates to attack one another, and find ways to avoid discussion of the policy issues the next president will face, the networks do the public a profound disservice. There is little doubt that while some networks and moderators did better than others in the 2008 primary season, on the whole, they failed.

METHODOLOGY

We coded all nationally broadcast primary debates in which a majority of the active candidates in a party participated, a total of 31 debates:

DatePartyNetwork
April 26, 2007 Democratic MSNBC
May 3, 2007 Republican MSNBC
May 15, 2007 Republican Fox
June 3, 2007 Democratic CNN
June 5, 2007 Republican CNN
June 28, 2007 Democratic PBS
July 23, 2007 Democratic CNN
August 5, 2007 Republican ABC
August 7, 2007 Democratic MSNBC
August 19, 2007 Democratic ABC
September 5, 2007 Republican Fox
September 9, 2007 Democratic Univision
September 26, 2007 Democratic MSNBC
October 9, 2007 Republican MSNBC
October 21, 2007 Republican Fox
October 30, 2007 Democratic NBC
November 15, 2007 Democratic CNN
November 28, 2007 Republican CNN
December 9, 2007 Republican Univision
January 5, 2008 Republican ABC
January 5, 2008 Democratic ABC
January 6, 2008 Republican Fox
January 10, 2008 Republican Fox
January 15, 2008 Democratic MSNBC
January 21, 2008 Democratic MSNBC
January 24, 2008 Republican MSNBC
January 30, 2008 Republican CNN
January 31, 2008 Democratic CNN
February 21, 2008 Democratic Univision/CNN
February 26, 2008 DemocraticMSNBC
April 16, 2008 DemocraticABC

We divided the questions into the following categories:

Substantive questions: These were legitimate inquiries about the candidates' records, opinions, and positions on policy issues and the next presidency.

Example: "I'm sure you'll agree there are a lot of beautiful young people out here in the audience today, and we're very pleased to be here at Howard University. So you can imagine how disturbed we were to find out from the Centers for Disease Control that African-Americans, though 17 percent of all American teenagers, they are 69 percent of the population of teenagers diagnosed with HIV/AIDS. Governor and candidates, what is the plan to stop and to protect these young people from this scourge?" (Michele Martin, PBS, 6/28/07)

Non-substantive questions: These fell into a number of categories, as follows:

Candidate "gotchas"

Rehashing gaffes -- questions that forced the candidate to explain a previous controversial statement

Example: "Senator, I was in your court until a couple of weeks ago. How do you reconcile the campaign of credibility that you have when you've made those comments about what happened getting off the plane in Bosnia, which totally misrepresented what really happened on that day? You really lost my vote. And what can you tell me to get that vote back?" (Voter, ABC, 4/16/08)

Defend another -- questions that forced the candidate to defend the actions or words of someone else

Example: "Senator Clinton, your party's leader in the United States Senate, Harry Reid, recently said the war in Iraq is 'lost.' A letter to today's USA Today calls his comments 'treasonous,' and says 'if General Patton were alive today, General Patton would wipe his boots with Sen. Reid.' Do you agree with the position of your leader in the Senate?" (Brian Williams, NBC, 4/26/07)

Playing politics -- questions that accused the candidate of engaging in evasiveness, flip-flopping, harsh campaign tactics, etc.

Example: "Senator McCain let me just ask you, if I can, for a 30-second follow-up. Because some people say that you have stepped away from the plan that you supported with Ted Kennedy last year, and also from negotiations to come up with a new plan. Is that because comprehensive immigration reform is bad politics for someone running for president?" (Chris Wallace, Fox, 5/15/07)

Frivolous/irrelevant questions

Campaign politics -- questions about polling, voter appeal, electability, etc.

Example: "Senator Clinton, recent national polls indicate the majority of the general public has an unfavorable view of you right now, at this point in time. Why do you think Republicans are looking forward to running against you with so much zeal?" (Brian Williams, NBC, 4/26/07)

Personal/character -- questions involving character traits, lifestyle choices, etc.

Example: "Senator Thompson, reporters on your campaign say your stumbles on the Terry Schiavo case and on the issue of oil drilling in the Everglades show that you don't do your homework. The week after, though -- you took a week off after the last debate. Speech yesterday was about five minutes long. Some people say you're lazy, sir. How do you deal with that?" (Wendell Goler, Fox, 10/21/07)

Triviality -- questions utterly unrelated to any policy or personal issue

Example: "For many here, in New England, the answer to this next question may be the most important one you answer tonight. Red Sox or Yankees? Governor Richardson?" (Allison King, NBC, 9/26/07)

Other kinds of problematic questions:

Hypothetical questions -- questions that offered extreme or absurd hypothetical situations and asked the candidate to respond

Example: "The questions in this round will be premised on a fictional, but we think plausible scenario involving terrorism and the response to it. Here is the premise: Three shopping centers near major U.S. cities have been hit by suicide bombers. Hundreds are dead, thousands injured. A fourth attack has been averted when the attackers were captured off the Florida coast and taken to Guantanamo Bay, where they are being questioned. U.S. intelligence believes that another larger attack is planned and could come at any time. First question to you, Senator McCain. How aggressively would you interrogate those being held at Guantanamo Bay for information about where the next attack might be?" (Brit Hume, Fox, 5/15/07)

Quarrel -- questions that sought to instigate conflict and argument between the candidates

Example: "Senator Obama, you said on a TV interview just this past weekend you didn't believe that Senator Clinton was able to unite this country. Why do you believe she can't?" (Suzanne Malveaux, CNN, 11/15/07)

A substantive question could also be coded for Hypothetical or Quarrel, in which case it was categorized as "substantive but flawed."

[1] Under our coding system, a question could be non-substantive but still relate to some policy issue. For instance, Republicans were asked about the political consequences of having a pro-choice nominee (Rudy Giuliani), a non-substantive question (as it focused only on the horse race) that nonetheless touched on a policy issue (abortion).

[2] Because the Gallup questions were open-ended, they did not correspond precisely to our categories. For means of comparison, we combined the Gallup results on Iraq, terrorism, foreign aid, international issues, and conflict in the Middle East to compare to our Foreign Policy & National Security category, which includes questions about Iraq. For our Economy category, we combined responses under economy in general, high cost of living, unemployment, lack of money, and wage issues. The one change across the three polls was that the economy rose in importance between the second and third polls, while foreign policy declined between the second and third polls.

[3] PBS also hosted a Republican debate on September 27, 2007. Because John McCain, Rudy Giuliani, Mitt Romney, and Fred Thompson did not attend, we chose not to include this debate in our data. But with the exception of an introductory question asking the attending candidates “why you chose to be here tonight and what you say to those who chose not to be here to-night,” all the questions in that debate were also policy-oriented.

[4] For that debate, we assigned the questions asked by Jorge Ramos of Univision to the Univision figures, and the questions asked by Campbell Brown and John King of CNN to the CNN figures.

[5] Since a number of journalists moderated one debate or asked a few questions in more than one debate, this chart includes only those who asked a total of 50 questions or more. All the journalists included in this chart moderated at least two debates.