Kudos to the Institute of Policy Studies for conducting a post-election survey to understand Singaporean voting behavior and releasing the digested results to the public. Alright then, with that short congratulatory note connoting respect for a worthy effort, here comes the critical blows.
Highlights of the survey are published on the front page of the Straits Times, 3 June 2006, with headlines blaring ‘Bread-and-butter issues “not the main concern” of voters’, and just slightly softer, ‘Need for efficient govt, fairness of policies mattered more to them: survey’. Lower down, a table ranked ‘What Mattered to Voters’ according to the percentages of those who responded with ‘very important’ and ‘important’ to the issues is presented for easy consumption (see figure below). The impression given is that there is a pecking order of issues the voters were concerned with. It presents a picture of a citizen who is more concerned with the efficiency and fairness of the government over democratic processes, and with personality over socioeconomic ‘bread-and-butter’ issues.

In the text, it is claimed that the study ‘found that, contrary to popular perception, the post-1965 generation did not have a clear preference for a more diverse political system’ and that ‘in fact, a slightly higher percentage of voters born before independence wanted such diversity’. Another claim: ‘honesty, efficiency, commitment and fairness of candidates mattered more to voters than their credentials or party affiliation’.
This is a picture of the Singaporean voter as a conservative and superficial citizen who sees politics as a reality-TV clash of celebrity-like individuals rather than a clash of national issues concerning socioeconomic and political realities. And with them headlines shouting at you, it is easy to conclude that the PAP was elected because of its efficiency and fairness as the governing party and its ability to resolve ‘bread-and-butter’ issues successfully. Thus, the Straits Times claims, ‘The survey resonated with the results of the election, in which the PAP garnered 66.6 per cent of the valid votes’ (the paper forgets to explain how).
Wait, not so fast. We need to look closer at the survey results. A triple whammy of authority (news media, experts and surveys) can lull us into a comfortable sense of fatalistic or ignorant political apathy: ‘oh why bother if the majority of Singaporeans are idiots’ (not true), ‘oh why bother if the government takes good and efficient care of us’ (subjective and one-dimensional). This is not to say that the news media and experts are intentionally tricking us. The key point is this. Surveys are highly contingent and heavily qualified representations of facts, but they are often presented to the public as absolute truths that are more newsworthy and soundbitey than saying ‘maybe lah’ (often on the untested assumption by media and experts that members of the public are consumerist simpletons).
They are several methods of getting at an understanding (note: not knowledge of the actual truth) of voting behavior. The survey-statistical method is one (an example of alternative methods is the comparative-deductive one I have employed in my analyses in the last few blog entries). The key characteristic of the survey method is its reliance on several statistical-mathematical assumptions and axioms that allow one to ask, say in the IPS survey, 985 people some questions to derive a calculated, approximated and inferred estimate (note: not exact and revealed truth) of the answers of over a million citizens eligible to vote. Two important implications follow.
First, with every result, there is always a margin of error, a plus/minus value that should be stated. That is why if you look at results in, say, US political opinion polls, there is usually a footnote stating the margin of error (e.g. +/- 3% etc.). Second, one can only say with a degree of certainty or confidence that the result with the margin of error can be generalized to the million citizens (one common standard in academic studies is 95% confidence, which means that there will be a 5% chance that the actual results of the million citizens do not fall within estimates). That is why sometimes polling results can go terribly wrong, for example, when the American news media jumped the gun and declared Al Gore the winner of the 2000 presidential elections on the basis of an estimate of the Florida returns before it was completely counted. The media forgot that it could only say with 90% or 95% or 99% that Al Gore was the winner and declared him to be 100% the winner. The two are related, if one wants a greater degree of certainty, the margin of error would be greater, thus a 95% degree of confidence has a greater margin of error than a 90% degree of confidence.
The problem with the presented and reported IPS survey is that we do not know the margins of error and the confidence level of generalization. Why is this important? It matters because the presentation of issues Singaporean voters are concerned in a tabular pecking order form becomes highly problematic and misleading if taken on face value. In the table reported in the papers and in the IPS presentation slides, it is implied that the ‘Personality of candidates’ with 77% saying ‘important’ and ‘very important’ is a more important issue than ‘Cost of living, including health care’ with 75%. Now, if the margin of error is 2% (polls often have 2% to 5% margins of error), the estimated range is 75% to 79% say personality is important and 73% to 77% say cost of living is important. This means that it may very be that voters actually consider cost of living to be more important than personality, so the results should be reported as ‘too close to call’.
If the margin of error is really 2%, which is a very forgiving guess on my part, it is simply wrong for the Straits Times to blare the headlines it did. ‘Efficient government’ seems to be indeed the most important issue for the voters with 93% saying so. The next issue with the highest percentage is ‘Different views in Parliament’ with 83%. Efficiency trumps democracy, but only if the margin of error is less than 5%. But fine, let’s go with the forgiving guess of 2% and conclude that efficiency is indeed the most important. In this case, it is misleading to say that ‘Need for efficient government, fairness of policies mattered more to them’ and to put policy fairness as second rank in the table because the latter, policy fairness (82%) is too close to call as a second most important issue when compared with ‘Different views in parliament’ (83%) and ‘Checks and balances in Parliament’ (82%).
Besides, if we forget about the margin of error for just a moment, why should ‘Fairness of government policy’ be in second place when it has the same percentage as ‘Check and balances’ and when it is lower in percentage as ‘Different views’? If it is because more voters say policy fairness is ‘very important’ (39%) compared to different views (38%), then surely checks and balances should be in second place when the two issues have the same percentage but checks and balances have more voters saying ‘very important’ (41%). It is simply illogical. The only explanations I can think of in placing policy fairness as second rather than the other two issues are, one, the aesthetics of putting issues concerning ‘government’ and ‘parliament’ next to each other, or, two, the downplaying of the democratic aspirations of Singaporeans. The second is obviously more plausible and probable. Why? Because it contradicts what the PAP governments have been saying all along, that is, the claim that Singaporeans do not want democracy just efficient and fair government, that only a vocal minority and not the silent majority want democracy.
Well, dead wrong. Singaporeans want efficient and fair government AND democracy. With the margin of error factored in, it cannot be said that Singaporeans want efficient and fair government more than democratic change. Besides, it is fallacious to present the two as mutually exclusive and contradictory choices, and the IPS poll suggests that Singaporeans is smart enough to know this and therefore choose to want both efficient and fair government and democracy. Which is why I am seriously perplexed that the IPS experts put ‘Desire for good governance, efficiency with the ideals of accountability and fairness reiterated’ as their conclusion but did not say a single word about the desire for democratic change in the conclusions of their presentation slides. Is this an abdication of responsibility to tell our leaders unpleasant representations and approximations of the truth, that they should not take the election results as a ‘mandate’ to continue business-as-usual?
Back to the screaming headlines. Bread-and-butter socioeconomic issues not ‘main concern’? Again, misleading. First, how shall we define ‘main concern’? ‘Main concern’ when it is right on top of the table or if it is above a certain percentage? In the implied IPS and Straits Times ranking, ‘Cost of living’ is sixth, but is it really only the sixth important issue? Cost of living has 75% saying it is important. With a 2% margin of error, it is too close to call that the ‘Personality of candidates’ (77%) is more important. And the difference between cost of living and the cluster of policy fairness, different views and checks and balances (82% to 83%) is 8%, which means that the actual margin of error must be less than 4% for one to say with a specified degree of confidence that it is not the SECOND most important issue after efficient government. Besides, isn’t 75% of the voters saying cost of living is an important issue significant in itself to warrant it being a ‘main concern’? The headline screaming at me in my slumbering consciousness in my pre-caffeine morning is worse than bad journalism. It is so misleading, it borders on unethical. Look down the table, 50% of the voters say their job situation is an important issue. Half of the population is feeling some anxiety about employment and three-quarters about costs of living and the headline tells me bread-and-butter is not ‘the main concern’!
Back to the IPS presentation slides. The survey suggests that 23.2% of post-65ers (21-39 years of age) are ‘conservative’ (read support/vote PAP), 44.4% are ‘swing’, and 32.4% are ‘pluralist’ (read support/vote opposition parties). For the pre-65ers, 29.7% are ‘conservative’, 35.7% ‘swing’, 34.6% ‘pluralist’. Based on this result, the Straits Times claims that the popular perception of the liberal and democratic aspirations of post-65er is wrong and that actually the pre-65er want more ‘diversity’. Misleading. First, the categories of political orientation seem to be downplaying democratic desire. The conventional contrast is ‘conservative’ versus ‘liberal’. ‘Pluralist’ does not have the same democratic connotation as ‘liberal’. Thus, the Straits Times write that it is ‘diversity’ in the political system that is wanted, which is all fine and dandy in matching the Prime Minister’s call to his PAP members of parliament to speak up in parliament to represent diverse views and the broken PAP record that sings about how diversity can be achieved with PAP backbenchers and undemocratic nominated MPs. So then, it is not democracy and opposition parties in parliament to check and balance the PAP that Singaporeans want, but the wooly, fuzzy, feel-good thing called ‘diversity’.
Second, the margin of error factor renders the results irrelevant. Assuming the forgiving 2% margin again, it cannot be said that pre-65ers or post-65ers want democratic change more than the other, the difference being only 2.2%. In fact, it is more accurate to say that the pre-65ers are more conservative than the post-65ers (difference of 6.5%) and that the post-65ers are more undecided than pre-65ers (difference of 7.7%), as both require a margin of error of less than 3% while the Straits Times claim require a margin of 1%. By claiming the first to supposedly debunk a popular belief rather than the more accurate latter two that provides more support for the popular belief, the Straits Times is again downplaying the desire for democratic change. Well, it is not ‘contrary to popular perception’ since nothing has been proven or even suggested. In fact, the poll suggests that both pre-65ers and post-65ers want democratic change EQUALLY, and that less post-65ers do not want democratic change, and that in any case, more Singaporeans want democratic change than those who do not want.
Through this whole critique, I have not dealt with the confidence level of the stated results, and as I have mentioned in the beginning, this is important because the confidence level determines the magnitude of the margin of error. At most, the results of the IPS survey can be stated with 99% confidence, but this will mean a large margin of error that will most probably render insignificant the ranking of the issues and the pre-65er and post-65er comparison. Nevertheless, it is significant that the IPS survey results as captured by the presentation slides and the way the Straits Times reported the survey are downplaying the desire for democratic change and the significance of socioeconomic concerns of Singaporeans. This is significant because the two mainstream opposition parties, the Singapore Democratic Alliance and the Workers’ Party, represent Singaporeans on the two issues. Perhaps more than another sad episode to add to my stock of low confidence in presentations of representations of truths by experts and the media alike, it represents some form of damage control of the results by vested conservative interests.
See Yawning Bread's excellent report on the IPS forum where the survey results were released.
See also the guest article by BL from Sg Entrepreneur on the survey results. BL correctly notes that the sample size is too small to rule out systematic bias. It is not quite correct to say that 5% of the population (around 50,000 in this case) need to be surveyed to rule out this systematic bias. Increasing the sample size tends to reduce the margins of error and therefore allowing one to increase the confidence level of making generalizing conclusions. Systematic bias will therefore be reduced, allowing one to see the results clearer without all them 'too close to call' scenarios and with more confidence that they are representative of actual reality. Nevertheless, BL makes a good point about the audacity of making bold conclusions on the basis of such a small sample. On the whole, the IPS survey is too close to call and thus has to be taken with doses of salt.
5 comments:
Indeed, other than margins of error there are many other methodological issues the report did not address adequately.
If the questions were reproduced verbatim in the report, one needs to ask about their criterion validity. For instance, the definition of necessary info to vote was not provided. Also, if it was a single-item measure, the reliability of the item is also questionable. Multiple-item measures would also allow more powerful inferential statistics, such as multiple regression, to be used for a more thorough understanding of the impact of demographics on the various instruments.
Methodology aside, questions like the importance of "Fairness of Govt policy" surely deserves a follow-up: Is the Government fair?
http://www.ips.org.sg/events/post-election/Survey_Report_postelection.htm
The IPS has posted a massively update of their paper that do answer some of your concerns here.
About the margin of error issue, there is no need to state that as it is a quota sample, not a random sample. The updated document states very clearly that "no generalisation" should be made from this type of samples, unlike a larger simple random sample.
As such, there is no need to provide a margin of error. It would actually be worrying if they had provided one, since it would give the impression that their sample can be generalised.
anonymous:
Yes, indeed, it is an astonishingly flawed survey. I am beginning to wonder who the intended audience is, because if it is the government bureaucrats, surely they have enough statistics training to see the problems, and if it is the public, did the IPS consider, with all their observations on blogging and the elections, that they may get peer reviewed by bloggers?
akikonomu:
Interesting update, because it includes many more technical details, like significant tests of means. The clear statement that 'no generationalization' should be made is too little too late.
Too late: Why did not the IPS make this statement at the Forum when it released the results to the press and the public? In view of the media attention that such a survey will get, did IPS not think that it is important to state this in the very beginning?
Too little: By changing the pdf file at the Straits Times Interactive site and on their own website, is IPS doing enough to rectify the misleading conclusions published in the Straits Times it has caused, inadvertantly or otherwise? Should not the Straits Times and IPS publish a letter in the ST Forum to clarify and make known this point? Perhaps it will in tomorrow's papers...
Yes, the idea that the results are not generalisable is bizarre, absurd, or both, for a study affecting all Singaporeans. Why conduct a study just for the 986 people surveyed? By whose/what imprimatur should their opinions be the only ones IPS is concerned about? And were the conclusions from the study only for the 986 respondents?
Hopefully, this is merely a sloppy first attempt at understanding post-election sentiments, a pilot study if you will. The IPS can probably do a more thorough job given the national interest in this matter, certainly more than just providing statistical lip service.
One of the most informative reads i found online... eventhough its more than 2 years late, still very relevant today...
:) THANKS!
-SMU undergrad.
Post a Comment