Introduction
Almost two decades have passed since the phrase ‘deliberative turn’ (Dryzek 2000) was coined. Despite the increased level of attention given to deliberative democracy in both theory (Cohen 1997; Dahlberg 2007; Dryzek 2016; Gastil 2008; Goodin 2003; Gutmann & Thompson 2009; Habermas 1984; Mansbridge 1983; Mercier & Landemore 2012; Sanders 1997; Young 2002) and practice (Cappella, Price & Nir 2002; Chang, Jacobson & Zhang 2013; Fishkin 1995; Luskin et al. 2014; Stromer-Galley & Muhlberger 2009; Perrault & Zhang 2019), we still know very little about the empirical mechanisms behind deliberation. Many existing studies have applied a simple ‘input-output’ model to examine the effects of deliberation. This model treats deliberation as a ‘black box’ (Stromer-Galley & Muhlberger 2009), which focuses on the deliberation process as a whole and is primarily interested in the outcomes that the whole deliberation process brings about. Although this approach contributes to the understanding of deliberation’s overall effects on psychological and behavioral outcomes, it fails to clearly differentiate the specific mechanisms in deliberation as well as their corresponding effects. It is thus imperative to open the ‘black box’ now to examine how the different components of deliberation work. In this article, we first categorize deliberation into two types, namely intrapersonal and interpersonal deliberation. Moreover, this study tries to decompose intrapersonal deliberation into two components: knowledge access and reflection.
In practice, deliberation experiments usually include a stage before the interpersonal deliberation starts, which allows participants to access background knowledge about the deliberation topic (e.g., Fishkin 1995; Luskin et al. 2014; Stromer-Galley & Muhlberger 2009; Perrault & Zhang 2019). Scholars proclaim that such background knowledge will help participants to enter deliberation with informed opinions. Very few studies, however, specifically focus on this first stage of intrapersonal deliberation (see Kim 2016 for an exception). Moreover, several scholars belittle the value of intrapersonal deliberation, regarding it as merely information processing (Weinmann & Vorderer 2015) or as a process ‘of inferior quality compared to public deliberation’ (Weinmann 2016: 8), because ‘it is usually deliberation with others, rather than solitary reasoning, which will have the desired transformative and epistemic properties’ (Landemore & Mercier, 2012: 920).
We argue that intrapersonal deliberation not only serves as an information acquisition session, in which accessing knowledge is deemed as the primary activity according to previous studies (Luskin et al. 2014; Stromer-Galley & Muhlberger 2009), but also triggers self-reflection (e.g., Fournier et al. 2011), which is essential to the idea of deliberative democracy. We believe that investigating two salient components in intrapersonal deliberation, namely knowledge access and reflection, is especially relevant to the current digital age, in which information is abundant but thoughtful consideration seems lacking. Civic technologies, at least in our view, can make both the implementation of each component and the examination of the mechanism of each feasible. In order to achieve different goals and cope with distinct contexts, we could use the appropriate technologies for these two components accordingly. Although knowledge access is a mechanism in which information flows from outside into deliberators, reflection triggers deliberators to think about their existing knowledge and prepare their opinions for future expression to the outside world. With careful reflection on their original positions, participants integrate knowledge into their own opinions; this could help in forming a more thorough judgment on the deliberation topic (Mansbridge 1983; McLeod et al. 1999). Hence, we define intrapersonal deliberation as an internal communication process, different from the interpersonal deliberation that happens with the presence of other people and through exchanges of views among individuals.
Intrapersonal deliberation differs from interpersonal deliberation, firstly because the latter requires the presence of other people. Intrapersonal deliberation, in contrast, could be implemented in a solitary state with the help of civic technologies. We are not saying that these two components, that is, knowledge access and reflection, never occur during interpersonal deliberation. Obviously, people could get knowledge from other people and be motivated to rethink their own views when encountering disagreements (e.g., Zhang & Chang 2014; Zhang 2015). A second difference is, however, that interpersonal communication achieves the two components through gives and takes of views and reasons among individuals. In other words, without a communication process between people, interpersonal deliberation cannot trigger knowledge access and reflection. For that reason, intrapersonal deliberation can be found in settings that do not involve direct personal interactions, such as when citizens encounter media information in their daily lives. In the present study, we try to use civic technologies to trigger knowledge access and reflection when participants are alone by themselves.
In this study, we conducted a between-subjects experiment (N = 83) to examine the effects of knowledge access and reflection in intrapersonal deliberation. A simple computer-based civic technology, providing information and asking participants to answer questions on a Webpage, was used to facilitate intrapersonal deliberation. By consequence, we looked at three sets of variables as effects: attitudes, attitude certainty, and willingness to express opinions. We found that knowledge access significantly changed attitudes but lowered the intention of future opinions expression in public, while reflection increased the level of perceived attitude correctness, a dimension of attitude certainty (Petrocelli, Tormala & Rucker 2007).
Literature Review
Why intrapersonal deliberation?
Goodin and Niemeyer (2003) suggest that deliberation is composed of discussion and consideration, and discussion is not usually seen as an end but a means to consideration. They argue that consideration should come earlier than discussion. It is impossible to have interpersonal deliberation if participants do not possess a basic understanding of the deliberation issue. In other words, these prior ideas and information constitute the ‘raw materials’ for later interpersonal discussion. Hence, intrapersonal deliberation, a process to reflect, consider and prepare one’s positions before discussion, is necessary in the whole deliberation process. Without intrapersonal deliberation, interpersonal deliberation is basically relying on a ‘polarized symbolic attitude’ (Goodin, 2008: 59) based on biases and myths rather than on factual information and careful considerations to exchange arguments, which hardly lead to successful decision making.
Information acquisition, or knowledge access, is thus important in the whole deliberation process. Using a field study, Goodin and Niemeyer (2003) compared attitude changes during the information phase and the discussion phase. They also asked participants to rank sources in terms of perceived impacts on their attitude changes. They found that from both sets of data, it is the information phase rather than the discussion phase that changes attitudes the most. This evidence supports the significance of information in deliberation.
In addition to this, opinion quality1 matters for deliberation scholars, because it is believed that pursuing reasoning to gain a prudential opinion is a crucial criterion for deliberation (Aristotle 1991; Cohen 1997; Dahlberg 2001; McBride 2003; Mercier & Landemore 2012). The Elaboration Likelihood Model (ELM) suggests that a higher level of political knowledge also brings a prudential view as well as a long-lasting attitude (Petty & Cacioppo 1986). Before deliberation, individuals might not have the chance to be involved in an intensive process of attitude formation. They merely rely on peripheral routes to decide their positions. The ELM suggests that a stronger ability, or the ‘requisite knowledge and opportunity to consider the arguments impartially’ (Petty & Cacioppo 1986: 19), predicts the pursuit of central routes to deliberate in attitude making, which might change attitudes and improve the quality of attitudes. These attitudes might become more stable, easier to access, and possess higher levels of confidence. Directly providing knowledge access in intrapersonal deliberation would serve this purpose.
Another indispensable component in intrapersonal deliberation is reflection, which is necessary to the consideration part of deliberation. Chambers (2003: 308) characterized deliberative democracy as a ‘talk-centric’ rather than a ‘vote-centric’ view of democracy. Talk contains an exchange of reasons for or against other positions, in which justification is necessary. On the other hand, listening is also indispensable in allowing talk to take effect (Dobson 2014). After listening, reflection is required to scrutinize the proposals and positions one heard. Hence, it is believed that justification and reflection are both essential for deliberation (Dryzek 2016).
Although reflection could be triggered by a social process of listening to others, it could also be implemented internally or individually. For example, Goodin (2000) suggests the method of ‘deliberation within’: people never use complete syllogisms in conversation, as they always assume that others are trying to make meaning. This shows that we understand other people mainly by imagining others’ voices inside our own heads. Thus, it is also plausible to artificially project ourselves to others and to ask ‘what would I say?’ to construct an imaginary face-to-face deliberation inside our heads. The imagination of others would allow members of the wider society to be ‘communicatively present’ in one’s mind, which makes intrapersonal deliberation possible.
In reality, knowledge access might also contribute to reflection by triggering people to think about their own opinions. However, conceptually, they are different and by treating them as two independent components, we can better open the ‘black box’ of deliberation and understand the mechanisms working behind the process. Moreover, in the digital age, we can design civic technologies that implement and examine knowledge access and reflection separately. In particular, some human-computer interaction (HCI) technologies could stimulate participants to carefully think about their own opinions, consider the reasons behind them, and make a thorough judgment. Hence, we believe that it is necessary both theoretically and practically to distinguish these two components, which we will further discuss in the next two sections.
Knowledge access and intrapersonal deliberation
Muhlberger and Weber (2006) found that discussions in interpersonal deliberation do not significantly improve participants’ knowledge, whereas reading and contemplating the knowledge materials do. Their results suggest that to foster an informed opinion, mere interpersonal deliberation is not enough. Deliberation scholars usually add a ‘library session’ (Stromer-Galley & Muhlberger 2009) or ‘briefing material’ (Fishkin 1995) before interpersonal deliberation. They believe that this would provide basic background information and enhance participants’ knowledge of the deliberation topic.
One important reason to provide information access is to enhance the opinion qualities of participants. In addition to the theoretical reasons mentioned in the previous section, in a laboratory experiment, Eveland and Schmitt (2014) found that the number of stories one reads positively predicts the amount of factual knowledge and the ability to build connections between different issues and actors. Recognizing the interrelations among ideas is deemed to be an antecedent of reasoning (Graber 2001: 14).
In addition to this, knowledge access also contributes to the societal-level democratic merit of deliberation. One major reason to pursue deliberative democracy is that it is more inclusive than aggregative democracy (Goodin 2003; Young 2002), which merely relies on voting and could lead to ‘the tyranny of the majority’ (Tocqueville 1996/1835: 413). By contrast, deliberative democracy includes the arguments of marginalized groups into its overall consideration. An equal presence in deliberation, however, never equates to equal participation (Sanders 1997), and there are asymmetrical power relationships in the deliberation process (Dahlberg 2007). There are special people with more political knowledge, or more usage of the media as a proxy for gaining political knowledge, who are more likely to engage in political discussions and other political participation activities (Cappella et al. 2002; Delli Carpini & Keeter 1996; Ho & McLeod 2008; Jung, Kim & de Zúniga 2011; Kim, Wyatt & Katz 1999). Political knowledge provides the facts that make participants aware of the relevance of politics to their lives and the opportunities to participate in politics (Delli Carpini & Keeter 1996: 224–225). Deliberation, adopting the norms from modern Western scientific debate, usually would privilege formal speech (Young 1996), which would disadvantage those marginalized groups with little political knowledge. Hence, we could expect that providing knowledge would help with this problem.
In empirical studies, Lee (2009) analyzed data from a representative survey in Hong Kong, which found that political knowledge correlates with the frequency of interpersonal political discussion. Associational relationships, however, could not clarify the causal order between political knowledge and political talks. It also ascribes the realm of political expression as a private conversation between acquaintances, which fails to examine other types of opinion expression, such as interpersonal deliberation. Kim (2016) conducted an experiment, which asked participants to attend a deliberation after processing background information materials. She found that participants processing high-information materials expressed more opinions than those processing low-information materials. A structured experimental design, however, recruits participants who are willing to attend a deliberation; this might not have captured the influence of information on ordinary participants who are originally reluctant to talk. In short, we still lack a thorough understanding of knowledge in intrapersonal deliberation.
Reflection and intrapersonal deliberation
As we mentioned before, deliberation is not a debate that aims to persuade others. It requires reflection. This process is different from knowledge access, which aims to provide factual information. The goal of reflection is not to change the amount of information one possesses, but to scrutinize the original ideas and prepare a quality opinion for its expression to the outside world.
Very few empirical studies in political communication and deliberation however, paid attention to reflection. One exceptional study (McLeod et al. 1999) used survey data, measuring reflection as respondents’ thinking, recalling, and following up on political information from the media, and found that participants who tend to reflect on issues are more likely to speak in an online forum. They explained that reflection contributes to the consolidation of information learned from the news. Mediated by political knowledge, more reflection is thus associated with a higher level of forum participation. This study partially endorsed the significance of reflection. It, however, failed to infer a causal relationship. Furthermore, this study defined reflection merely as the extent of processing information obtained from the mass media, which did not have a strong relationship with deliberation. Perhaps it just reflected interest in the topic or political sophistication. On the contrary, Weinmann (2018) adapted the definition of ‘private deliberation’ from Mercier and Landemore (2012), who gauged the ‘deliberation within’ or the level of reflection by several criteria: the simulation of divergent opinions, the acquisition of arguments for these opinions, and their evaluation. Weinmann (2018), however, just constructed a scale with several items for gauging reflection, while this scale hardly contributed to the examination of a causal relationship. To investigate the causal effects of reflection, it is necessary to conduct experiments. As previous studies have, however, primarily relied on survey items to measure reflection, very few studies have developed a manipulation on reflection in intrapersonal deliberation. Hence, this study will develop a design to trigger reflection in intrapersonal deliberation, which will be discussed in the next section.
A civic tech design of reflection in intrapersonal deliberation
Previous studies have used several methods to provide knowledge access (e.g., briefing materials in a deliberative poll, Fishkin 1995), but they rarely offer a reflection session. We believe that a civic tech design can help with stimulating and testing reflection in intrapersonal deliberation. The inspiration for the reflection design in this study, which is operationalized as answering open-ended questions, mainly comes from the literature on education and HCI (e.g., Xiao et al. 2015; Menon, Zhang & Perrault 2020). Education scholars have studied reflection for a long time, especially through the form of allowing students to provide several outputs to enhance their learning performances. For example, Schaffer and Hannafin (1986) found that adding multiple choice questions in the instruction video enhanced scores in the recall test. Asking students to generate questions on the material they process also leads to gains in comprehension (Rosenshine, Meister & Chapman 1996). In addition to this, with the rise of online education, nowadays HCI scholars increasingly try to use reflection to enhance the efficiency of remote learning. For example, in addition to simple quizzes embedded in interactive instruction videos, scholars design interactive learning systems to ask students to provide audio explanations (Kim et al. 2015) or textual reflections (Glassman et al. 2015). It is found that students like these designs, which ask them to explain their processes of solving problems or to write texts about their own understanding of the instruction materials. Although these studies usually provide other complicated interaction forms, such as interpersonal feedback, and merely measure the affection shown toward the design, we could still borrow the ideas of using relatively complex forms of feedback, instead of simple forms like answering objective quizzes, to let participants be more mindful about their opinions and to trigger reflections.
Several theoretical considerations predict the effects of this reflection design. Pingree (2007) summarized three types of expression effects in deliberation. Although he built the model based on interpersonal deliberation, some mechanisms would still work in intrapersonal deliberation. First, people generally have the motivation to perform well during the deliberation process, the expectation effect, which indicates that awareness of the future action to write a response to received content enhances the attention paid to and cognitive efforts spent on the provided knowledge materials in intrapersonal deliberation. This could lead to a higher level of knowledge processing during intrapersonal deliberation. Meanwhile, the composition effect shows that attitude articulation would increase the accessibility of an attitude because expression itself uses working memories, which indicates that at least reflection would change the accessibility of attitudes. In addition, as a persuasive expression usually is more likely to be public-spirited, a motivation to perform well in expression would encourage people to write more public-spirited responses. Derived from self-perception theory (Bem 1967), which mentions that people would infer their attitudes from their behaviors, a more public-spirited expression has the potential to form a more public-spirited attitude during this whole process.
Empirically, generation effect research in educational psychology (Slamecka & Graf 1978) shows that content generated by the subjects themselves enhances memory over just being presented with content. A specific instantiation further shows that asking participants to simply give answers to the question ‘why’ to explain stimulus information (Pressley et al. 1987) contributes to recall. Although these studies tested the memory effects of answering questions, very few studies looked at attitudinal effects in a political context. One exception by Fournier et al (2011) found that a simple design, which forces participants to consider multiple dimensions of the issue by asking them to finish multiple choice questions in a questionnaire, would lead to a higher-quality attitude, which is more consistent and reliable.
In summary, we could expect that reflection, allowing participants to respond to information and questions, would significantly change the attitude as well as relevant attributes of the attitude. We, however, still lack empirical evidence to directly support this statement.
Attitudes, attitude certainty, and willingness to express opinions
To gain a thorough understanding of knowledge access and reflection in intrapersonal deliberation, we expanded the measurements of effects, which previously mainly looked at attitude, to include attitude certainty and willingness to express opinions into our examination. We expect that both knowledge access and reflection might alter attitudes and relevant attributes of their attitudes, as well as change their willingness to express their views later in interpersonal deliberation.
Former studies have provided abundant evidences that interpersonal deliberation can change the political attitude of participants (e.g., Gastil, Black & Moscovitz 2008). We can expect that intrapersonal deliberation, which shares several key components with interpersonal deliberation such as knowledge gains and reflection on one’s original opinion, could also lead to similar effects. Especially as, the level of political knowledge among average citizens is often low (Delli Carpini & Keeter 1996), which opens the possibility of attitude change through information gain. Moreover, reflection is even rarer in everyday life and if given the opportunity to reflect on one’s opinion, one might change his/her attitude to the one with better justifications. Thus, we hypothesize that the two different variables in intrapersonal deliberation, namely knowledge access and reflection, would change political attitudes.2
H1a: Subjects with access to knowledge materials will differ from those without knowledge access in their attitudes.
H1b: Subjects who answer reflection questions will differ from those who do not answer reflection questions in their attitudes.
Attitude certainty indicates ‘the sense of conviction with which one holds one’s attitude’ (Tormala & Rucker 2007), which may refer to the strength of the attitude, and is regarded as one important criterion to measure the quality of opinion (Goodin 2003; Price & Neijens 1997). Specifically, there are two different dimensions within attitude certainty (Petrocelli et al. 2007): attitude clarity, reflecting the feeling that one knows one’s true attitude toward the topic, and attitude correctness, referring to the confidence that the attitude is correct. Attitude certainty would benefit the deliberation process because it would encourage information sharing behaviors (Cheatham & Tormala 2015), motivate participants to express their opinions even when they are minorities and facing a ‘spiral of silence’ (Matthes, Morrison & Schemer 2010), and might further predict active participation in interpersonal deliberation.
Although very few studies look at attitude certainty in deliberation, psychological empirical studies provide possible links between knowledge, reflection, and attitude certainty. For knowledge access, providing information to participants would enhance perceived knowledge, which would raise the level of attitude certainty (Smith et al. 2008). In addition to this, the neutral information we usually use in the intrapersonal deliberation is two-sided, and previous empirical studies support the view that exposure to two-sided information would also increase attitude certainty (Rucker et al. 2008). For reflection, as we mentioned above, a generational effect and an expression effect resulting from answering questions might lead to a high-quality attitude, which is more accessible, consistent, reliable, and certain. Based on the analysis above, we can propose hypotheses between knowledge, reflection, and attitude certainty.
H2a: Perceived correctness of attitudes would be greater for subjects with access to knowledge materials than those without.
H2b: Perceived correctness of attitudes would be greater for subjects who answer reflection questions than those who do not.
H3a: Perceived clarity of attitudes would be greater for subjects with access to knowledge materials than those without.
H3b: Perceived clarity of attitudes would be greater for subjects who answer reflection questions than those who do not.
As a component distinct from interpersonal deliberation, it is necessary to test the effects of intrapersonal deliberation on the willingness to express opinions, which predicts participation in future interpersonal deliberation and other civic engagements. Despite the importance of willingness to express opinions, most deliberation studies have not tested it as a dependent variable but treated it as the starting point of the interpersonal deliberation they examined. In contrast, the variable has been widely examined in studies on spiral of silence (e.g., Hayes, Shanahan & Glynn 2001) and we gain insights from these studies. These indirectly related literatures, however, display ambivalent predictions. On the one hand, surging levels of perceived knowledge during intrapersonal deliberation would enhance the probability of expressing opinions (Cheatham & Tormala 2015). On the other hand, more elaborations on the opinion of the topic would make participants notice multiple facets of the topic and lead to ambivalences toward the topic, which may dampen the motivation for future civic engagement (Mutz 2002). As a significant component of political participation, we can expect that the willingness to express an opinion would also be influenced. Therefore, we propose two research questions about the relationship between knowledge access, refection, and willingness to express opinions.
RQ1: What is the relationship between knowledge access and willingness to express opinions?
RQ2: What is the relationship between reflection questions and willingness to express opinions?
Method
A between-subjects experiment (N = 83) was conducted to investigate the multiple cognitive effects of knowledge access and reflection in intrapersonal deliberation. Independent variables were manipulated by a 2 × 2 factorial design, and dependent variables and control variables were examined by an online questionnaire. A two-way multivariate analysis of covariance (MANCOVA) and a two-way analysis of covariance (ANCOVA) were employed as the main data analysis techniques, and a one-way analysis of variance (ANOVA) was used for the manipulation check.
Participants
A total of 83 undergraduate and graduate students in a local university in Singapore participated in the experiment in exchange for course credits or monetary compensation. Across the sample, 38 (46%) were male and 45 (54%) were female, with an average age of 21.67 (SD = 1.93). Each participant was randomly assigned to one of the four experimental conditions.
Research subjects were recruited through two approaches: the first one was to cooperate with an undergraduate module, which required students’ participation in relevant studies for credits, and the second one was to send out invitation letters via student email lists of multiple faculties and departments of the university. The recruitment criteria of being older than 18 years and being a local citizen were highlighted in invitation letters to avoid the inclusion of unqualified subjects. Multiple time slots of the experiment were also provided in the letters for potential participants to schedule their participation.
Experimental design
In this study, the independent variables were manipulated by a 2 (Reflection: reflection questions, no reflection questions) × 2 (Knowledge: briefing materials, no briefing materials) between-subjects factorial design (as shown in Figure 1). Knowledge access was operationalized as the briefing materials for subjects to go through. The briefing materials were designed as four-section printouts about the foreign workforce issue in Singapore. Reflection was manipulated on two levels as the existence (or not) of reflection questions to be answered by subjects at the end of each section of the materials. The answers of the reflection questions were inputted into the given webpage through a computer.
Experimental design.
The foreign workforce issue was chosen because it was one of the most prominent and controversial social issues in Singapore’s context (Zhang 2016). Statistics released by the Minister of Manpower in Singapore showed that, among Singapore’s total population of 5.47 million, 1.60 were foreigners without citizenship or permanent residency. The massive influx of a foreign workforce is widely regarded as an important driving force for local economic development, while it also creates discontent among locals on social issues such as housing, transportation, and employment.
The briefing materials and reflection questions were developed and structured based on the analytical model of deliberation proposed by Gastil (2008). Among the four parts of the slides, the first two parts were information based, mainly focusing on the background of foreign workers in Singapore and local-foreign relations. The third part introduced different viewpoints and values on this topic, including both positive and negative voices; the fourth part focused on related policy issues, such as housing policies and transportation policies. The reflection questions were designed accordingly to help participants to scrutinize the assigned materials and elaborate their own opinions. For example, the question of ‘What is your opinion on local foreign relations?’ was posed after the second part of the material, which provided information about local-foreign relations.
Procedure
The experimental research lasted for 4 days from March 8, 2016 to March 11, 2016. The experiment was conducted in a computer lab with access to the internet.
When a subject arrived at the lab, one experimenter firstly reconfirmed that he or she was a local citizen and older than 18 years. The experimenter then began briefing the subject about the research and obtaining his or her oral consent before randomly assigning him or her into an experimental group. After this random assignment, one experimenter led the subject to the right seat and set up the right experimental treatment for him or her according to the experimental group assigned. When a subject finished the first stage of the experimental treatment of reading briefing materials and/or answering reflection questions, one experimenter would help him or her to proceed into the second stage of the online questionnaire (except for subjects in group 1 who finished the questionnaire in the first stage). After completing the questionnaire, subjects who did not participate for module credits were compensated.
Measures
In the online questionnaire, a series of political psychological variables were measured as dependent variables (willingness for opinion expression, attitude toward foreigners, attitude certainty), control variables (issue interest, issue efficacy, political tolerance), and two variables for manipulation check (perceived knowledge and actual knowledge).
Attitude toward foreigners
Nine 5-point Likert items (1 = strongly disagree; 5 = strongly agree) were used to measure attitudes of local subjects to foreign workers in the Singapore context. Using eigenvalues higher than one and the varimax rotation method, factor analysis generated two separate variables out of these items: a six-item scale of tolerant attitude (Cronbach’s α = 0.87, M = 4.16, SD = 0.59) and a three-item scale of antagonistic attitude (Cronbach’s α = 0.73, M = 3.36, SD = 0.75) (see Table 1).
Factor analysis results of attitude items.
| Attitude items | Factor loadings | |
|---|---|---|
| Tolerant | Antagonistic | |
| Tolerant attitude | ||
| Singapore needs foreigners in our workforce | 0.81 | |
| Foreign professionals (e.g., managers, university lecturers, doctors, lawyers, auditors) make important contributions to Singapore | 0.75 | |
| Foreign mid-level skilled workers (e.g., nurses, technicians, physiotherapists) make important contributions to Singapore | 0.79 | |
| Foreign mid-level service staff (e.g., waiters, retail shop sales staff) make important contributions to Singapore | 0.79 | |
| Foreign unskilled workers (e.g., construction workers) make important contributions to Singapore | 0.73 | |
| Overall, foreigners make important contributions to Singapore | 0.81 | |
| Antagonistic attitude | ||
| There are too many foreigners working or living in Singapore. | 0.89 | |
| The number of foreigners who are working or living in Singapore is increasing too fast. | 0.90 | |
| The number of foreigners in Singapore has made me feel like a foreigner in my own country. | 0.67 | |
Attitude certainty
Two separate measures adapted from the study of Petrocelli et al. (2007) were used to examine subjects’ sense of conviction toward their own attitudes. Attitude clarity, which refers to the sense that one knows what his or her true attitude is, was measured by a 4-item, 9-point Likert scale (1 = not certain at all; 9 = very certain). Items included ‘How certain are you that you know what your true attitude on foreign workforce issue really is,’ ‘How certain are you that the attitude you expressed toward foreign workforce issue really reflects your true thoughts and feelings,’ ‘To what extent is your true attitude toward foreign workforce issue clear in your mind,’ and ‘How certain are you that the attitude you just expressed toward foreign workforce is really the attitude you have’ (Cronbach’s α = 0.89, M = 6.11, SD = 1.38). Attitude correctness refers to the sense of a person that his or her attitude is correct. It was measured by a 3-item, 9-point Likert scale (1 = not certain at all; 9 = very certain) with items of ‘How certain are you that your attitude toward foreign workforce issue is the correct attitude to have,’ ‘To what extent do you think other people should have the same attitude as you on foreign workforce issue,’ and ‘How certain are you that of all the possible attitudes one might have toward foreign workforce issue, your attitude reflects the right way to think and feel about the issue’ (Cronbach’s α = 0.88, M = 5.73, SD = 1.47).
Willingness to express opinions
Five 5-point Likert items (1 = strongly disagree; 5 = strongly agree) were used to measure subjects’ future intention and willingness to express their opinion about foreign workforce issues and participate in deliberative discussions. Using eigenvalues higher than one and the varimax rotation method, factor analysis yielded two distinct variables out of these items: a two-item scale of willingness to express opinions in private (Cronbach’s α = 0.80, M = 4.20, SD = 0.62) and a three-item scale of willingness to express opinions in public (Cronbach’s α = 0.75, M = 3.17, SD = 0.79) (see Table 2).
Factor analysis results of willingness items.
| Willingness items | Factor loadings | |
|---|---|---|
| Private | Public | |
| Willingness to express opinions in private (Private) | ||
| I am willing to voice my opinion toward foreign workforce issue in front of close friends. | 0.93 | |
| I am willing to voice my opinion toward foreign workforce issue in front of spouse/boyfriend/girlfriend. | 0.82 | |
| Willingness to express opinions in public (Public) | ||
| I am willing to voice my opinion toward foreign workforce issue in front of neighbors. | 0.63 | |
| I am willing to participate in discussion in public about foreign workforce issue | 0.86 | |
| Assuming an online policy discussion platform toward foreign workforce issue is available, I predict I will use it to express my opinion. | 0.84 | |
Control variables
Issue interest, issue efficacy, and political tolerance have been included as control variables.3 The interest of subjects in the issues about foreign workforce was measured by one 5-point Likert item (1 = strongly disagree; 5 = strongly agree) of ‘I am interested in foreign workforce issues’ (M = 3.65, SD = 0.71).
Political efficacy examines subjects’ trusts in government and their beliefs that they can understand and influence politics (Campbell, Gurin & Miller 1954). It includes two different components, internal efficacy, which focuses on citizen’s subjective competence to understand and participate in political affairs, and external efficacy, which concerns the responsiveness of the government to public participation (Morell 2003). Two scales adapted from the items developed by American National Election Studies (ANES) were employed to measure these two dimensions of political issue efficacy. Internal efficacy was measured by a 4-item, 5-point Likert scale (1 = strongly disagree; 5 = strongly agree) including items of ‘I consider myself to be well qualified to participate in foreign workforce issue,’ ‘I feel that I have a pretty good understanding of the foreign workforce issues in Singapore,’ ‘I feel that I could do as good a job in foreign workforce issues as most other people,’ and ‘I think that I am better informed about foreign workforce issues than most people’ (Cronbach’s α = 0.86, M = 3.05, SD = 0.76). External efficacy was measured by a 3-item, 5-point Likert scale (1 = strongly disagree; 5 = strongly agree) including items of ‘People like me do not have any say about what the government does on foreign workforce issues,’ ‘Public officials do not care much what people like me think about foreign workforce issues,’ and ‘Voting is the only way that people like me can have any say about how the government runs things on foreign workforce issues’ (Cronbach’s α = 0.73, M = 2.93, SD = 0.84).
Political tolerance refers to the willingness of citizens to tolerate offensive groups or ideas (Sullivan & Transue 1999). It was measured by a 5-point Likert scale of two opposite statements. Participants were asked to report their position between the two sides (1 = strongly agree to statement A; 5 = strongly agree to statement B). The statements were ‘Be able to prohibit the expression of beliefs and values that it feels are repugnant to the people’ (A) versus ‘Guarantee to all the right to express their personal beliefs and values, whatever they are’ (B) (M = 3.30, SD = 1.03).
Perceived knowledge
The level of self-perceived knowledge of subjects on foreign workforce issues was measured by a 3-item, 7-point Likert scale (1 = strongly disagree; 7 = strongly agree). Items included ‘If someone asks me about foreign workforce issue, I would have enough information to inform him or her,’ ‘I am knowledgeable about foreign workforce issue,’ and ‘I am confident about my knowledge about foreign workforce issue’ (Cronbach’s α = 0.97, M = 3.88, SD = 1.33).
Actual knowledge
The level of actual knowledge on the foreign workforce issue was gauged by a set of 4-item, multiple choice knowledge questions. For each question, we asked participants to select an answer from four options or reported ‘Don’t know.’ We calculated their numbers of correct answers to measure their actual knowledge (M = 1.35, SD = 0.93).
Results
Manipulation check
For the manipulation check of knowledge access, subjects who were exposed to briefing materials were expected to have higher self-perceived knowledge and a higher level of actual knowledge on foreign workforce issues than those who were not given any briefing materials. An ANOVA revealed that perceived knowledge varied significantly for different conditions of knowledge (F (1, 81) = 14.63, p < 0.001). As expected, subjects who had been exposed to briefing materials (M = 4.40, SD = 1.10) had higher perceived knowledge than subjects who did not go through any materials (M = 3.37, SD = 1.35). In addition, another ANOVA showed that the effects of knowledge access on actual knowledge approached the borderline significance (F (1, 81) = 3.38, P = 0.07). Those who had access to knowledge materials (M = 1.54, SD = 0.91) had a higher level of actual knowledge compared with those without (M = 1.17, SD = 0.92). Both results indicated the effectiveness of this experimental manipulation.
For the manipulation of reflection, subjects assigned to experimental groups with reflection questions completed the given four questions during the experimental process (average length of answers: 163 words). Their answers included the consideration and reflection for important information for decisions, first priority value in decision making and solutions on population issues. For example, one response tried to reflect on policy first from the perspective of local citizens. Later, it began to refer to the view of foreign workforces:
…reduce the foreign employment in Singapore in general, such that there are less foreign workers in Singapore competing with locals. I think they are fair to the locals… But I feel that foreigners deserve some sympathy as well, since they come to Singapore to work because of particular reasons… (Policy) might not be fair to them.
Effects on attitude
As the strength of correlation between the two dependent variables of attitude toward foreigners (tolerant attitude and antagonistic attitude) was low (Pearson’s r = 0.10, p = 0.36), two separate two-way ANCOVAs were carried out respectively to determine their relationships with the independent variables of knowledge and reflection. Issue interest, issue efficacy (internal efficacy and external efficacy), and political tolerance were controlled as covariates.
Results of ANCOVAs indicated a statistically significant main effect of knowledge on antagonistic attitude (F (1, 73) = 4.66, p = 0.03, partial η2 = 0.06), which suggested that subjects who were exposed to briefing materials showed lower antagonistic attitude toward foreigners than those who did not go through any materials (Δ M = 0.36, SE = 0.17, p = 0.03). No significant relationship between knowledge and tolerant attitude was identified. Hence, H1a was partially supported. Moreover, there were no significant main effect of reflection and interaction effect on either tolerant attitude or antagonistic attitude. So H1b was rejected.
Effects on attitude certainty
The two dependent variables of attitude certainty (attitude clarity and attitude correctness) were correlated (Pearson’s r = 0.53, p < 0.001), so a two-way MANCOVA was performed to test whether they had significant relationships with the independent variables of knowledge and reflection after controlling for issue interest, issue efficacy (internal efficacy and external efficacy), and political tolerance.
According to the results of MANCOVA, no significant main effect and interaction effect were found on the combined dependent variable of attitude certainty. Univariate analysis revealed that reflection significantly affected subjects’ perceived correctness of attitude (F (1, 75) = 5.62, p = 0.02, partial η2 = 0.07), while showing no significant effect on their perceived clarity of attitudes. In particular, attitude correctness was higher among participants who had answered reflection questions than those who did not (Δ M = 0.70, SE = 0.30, p = 0.02). Therefore, H2b was supported, whereas H3b was rejected. In addition, no significant interaction effect and main effect of knowledge were found on either attitude clarity or attitude correctness, so H2a and H3a were not supported.
Effects on willingness for opinion expression
As the two dependent variables of willingness for expression (willingness for opinion expression in private and willingness for opinion expression in public) were correlated (Pearson’s r = 0.48, p < 0.001), a two-way MANCOVA was conducted to explore their relationships with the independent variables of knowledge and reflection. Issue interest, issue efficacy (internal efficacy and external efficacy), and political tolerance were controlled as covariates.
Results of MANCOVA revealed a statistically significant main effect of knowledge on the combined dependent variable of willingness for expression (Wilks’ λ = 0.90, F (2, 74) = 3.91, p = 0.02, partial η2 = 0.10), whereas no association between reflection and willingness was identified. Moreover, interaction effect between independent variables on the combined dependent variable was not significant.
Univariate analyses indicated that knowledge only had significant effect on subjects’ willingness to express their opinions in public (F (1, 75) = 4.77, p = 0.03, partial η2 = 0.06). Specifically, subjects who were exposed to briefing materials were less likely to express their opinions in public as compared with subjects who were not given any materials (Δ M = 0.33, SE = 0.15, p = 0.03). No significant interaction effect was found on either of the dependent variables. A summary of results on hypotheses and research questions is shown in Table 3.
Summary for results of hypotheses and research questions.
| Hypothesis | Independent variable | Dependent variable | F value | p value | ∆ |
|---|---|---|---|---|---|
| H1a | Knowledge access | Antagonistic attitude | 4.66 | 0.03 | 0.36 |
| Tolerant attitude | 0.15 | 0.70 | |||
| H1b | Reflection | Antagonistic attitude | 0.08 | 0.78 | |
| Tolerant attitude | 1.44 | 0.23 | |||
| H2a | Knowledge access | Perceived attitude correctness | 0.85 | 0.36 | |
| H2b | Reflection | Perceived attitude correctness | 5.62 | 0.02 | –0.70 |
| H3a | Knowledge access | Perceived attitude clarity | 0.02 | 0.88 | |
| H3b | Reflection | Perceived attitude clarity | 0.46 | 0.50 | |
| RQ1 | Knowledge access | Willingness to express opinions in private | 0.41 | 0.52 | |
| Willingness to express opinions in public | 4.77 | 0.03 | 0.33 | ||
| RQ2 | Reflection | Willingness to express opinions in private | 0.05 | 0.82 | |
| Willingness to express opinions in public | 0.28 | 0.60 | |||
∆ denotes the post hoc result on the discrepancy between those without knowledge access/reflection and those with knowledge access/reflection. The bold p-values denoted that these corresponding effects were significant.
Discussion
This article examines the effects of knowledge and reflection in intrapersonal deliberation. The findings indicate that knowledge has changed political attitudes, specifically by reducing the antagonistic attitudes toward foreign workers. In addition to this, knowledge has reduced their willingness to express opinions in public, whereas reflection has enhanced the level of attitude correctness.
These findings show that both knowledge and reflection in intrapersonal deliberation are actually double-edged swords in the overall deliberation process. After providing participants with access to the background knowledge, attitudes on the topic changed in a relatively short period of time, which demonstrates the effectiveness and importance of intrapersonal deliberation. On the contrary, exposure to background knowledge surprisingly reduced the willingness to express opinions in public. This finding suggests the lower possibility to engage interpersonal deliberation for those with knowledge access. Perhaps the briefing materials had enhanced their perceived complex and controversial nature of the issue and made them hesitant to express their opinions in public, which might also prevent them from attending later interpersonal deliberation.
Interestingly, these results contrast with the understanding gained from a former structured experimental study (Kim 2016), which supported the view that a higher level of knowledge on the deliberation topic, obtained from intrapersonal deliberation, would contribute to engagement in public deliberation processes. Accessing knowledge would directly increase the quantities of expressions in interpersonal deliberation. Our study tells another story: although participants perceived a higher level of knowledge through accessing knowledge, the willingness and intention to attend an interpersonal deliberation actually became lower. It appears that the role of political knowledge in encouraging deliberation participation is much trickier than we previously thought. In the study by Kim (2016), it is possible that recruiting only those participants who already expressed willingness to join later discussions biased the results. For future studies, in addition to inspecting participants’ performances in interpersonal deliberation, we should also pay attention to this problem at the recruitment stage of many deliberation studies; this has often been ignored in structured experiments.
On reflection, we found that it can make subjects perceive their own attitudes as correct more strongly than those who did not get a chance to write reflective comments. Perhaps this is due to confirmation bias (Nickerson 1998), as people would actively seek and interpret evidence to support their beliefs. Generational effects (Pressley et al. 1987) indicate that by answering questions, the information supporting those positions would be more accessible in subjects’ minds. Hence, when they evaluate the correctness of their attitude, they could easily find supporting evidence to endorse their positions, which eventually increases the perceived attitude correctness. The implications of this finding are mixed: a higher level of attitude correctness after reflection might encourage information sharing and attitude disclosure (Cheatham & Tormala 2015; Matthes et al. 2010). Participants might be more willing to actively take part in interpersonal deliberation; active participation is a precondition of the success of the overall deliberation process. Empirical research, however, shows that attitude correctness will lead to a competitive conflict style of conversation, which might prevent properly addressing disagreements (Rios, DeMarree & Statzer 2014). As disagreement is regarded as a ‘core requirement’ in the deliberation process (Mutz 2008) and poor responses to disagreements could dampen deliberation (Mendelberg & Oleske 2000), attitudes with high correctness might cause real problems for deliberation.
In addition, a high attitude certainty usually predicts a strong resistance to persuasion (Petrocelli et al. 2007), because this attitude has dense connections with other attitudes and prior experiences so that new pieces of information generate a small alteration in this attitude. People might either actively seek a counterargument to refute the new information or selectively pay attention to the part of the new information that aligns with their prior attitudes (Eagly & Chaiken 1995). The implications for deliberation are two-folded. On the one hand, if they certainty denote that the opinion has well-rounded reasons and justifications behind it, the resistance to persuasion actually suggests that they have formed a prudential opinion, which is crucial to deliberation. On the other hand, if participants overlook those reasonable counterarguments, high certainty might prevent mutual understanding, as people who think their attitudes are correct might refuse to consider others’ perspectives.
There are several hypotheses that are rejected in this study. Knowledge failed to change attitude certainty, which contrasts with the association between the two found in previous studies (Smith et al. 2008). This suggests that we should further clarify the role of information and knowledge in shaping attitude certainty. Perhaps the amount of knowledge never equates to the comprehensiveness of knowledge. The attitudes of those who encountered well-balanced or two-sided information (which is our knowledge access design) might not show higher levels of certainties than those who encountered highly biased information with strong inclinations. The balanced knowledge may motivate them to think about competing views with both pros and cons. Future studies should try to test this difference between one-sided versus two-sided knowledge in the context of deliberation. Reflection had no effect on attitude clarity, but it did show an effect on correctness. Perhaps reflecting and internalizing other views might improve the sense of generalizability of the position, which predicts perceived attitude correctness. It, however, might not help to clarify one’s personal disposition, because it might be blurred with different views from the ‘communicatively present’ others. Future studies should distinguish these two constructs as well as corresponding functional differences. Finally, there are no significant differentiations on attitudes between people with and without reflection in intrapersonal deliberation, which coincides with the critique by Schneiderhan and Khan (2008). They found that using essay writing to trigger intrapersonal deliberation has no effect on the attitudes of participants. A mere internal reflection without any external stimulus might not be strong enough to bring out attitude changes.
In summary, this study appears to support the view that both knowledge and reflection in intrapersonal deliberation are indispensable components. They, however, work in different ways: without knowledge access, reflection would actually function as a self-reinforcement process, which reduces the level of uncertainty. It could cause several obstacles for interpersonal deliberation. This coincides with some critiques toward intrapersonal deliberation (Landemore & Mercier 2012; Schneiderhan & Khan 2008), in which confirmation bias (Nickerson 1998) would dampen reasoning. Participants would only find more evidence to support their original positions rather than reflect and accept new proposals. With the supplement of knowledge, intrapersonal deliberation can, however, change attitudes, which means it is not just a self-reinforcement process. Moreover, with the help of reflection design, participants become increasingly confident about their arguments; this indicates an improvement in the quality of attitude. It actually shows that intrapersonal deliberation can implement several tasks of reflection to scrutinize their original positions, which meets our expectation.
No interaction effects between knowledge access and reflection were found in this study. Although these interactions were not what we expected to see according to existing literatures, it might be worth discussing the lack of such findings. Firstly, there is a theoretical reason to believe that the two components are relatively independent of each other. People can access new information without reflecting on them. They can also question their own views without being given new information. There is no clear theoretical justification to say that their cumulative effects would be different from their individual effects. Secondly, our dependent variables might be responsive to influences from only one of the two independent variables. For instance, attitude itself is only changeable through knowledge access while attitude correctness is only changeable through reflection. Interaction effects might emerge if we look at other dependent variables that are responsive to both components’ influences. Lastly, we have to admit that it is possible that our sample size is not large enough to allow statistical significances of interactions effects to emerge, which leads to our discussion of limitations.
There are several limitations in this study, which could provide directions for future research. First, due to the nature of experimental design, we could not clarify the causal relationships among our dependent variables. Future studies should consider research designs that allow the examination of the causal relationships among attitudes attributes and willingness to express opinions, or take a further step to examine actual expressions. Regarding our experimental design, there is room for improvement as well. For example, more control variables such as personality traits could be included. The size of our participant pool can be increased to gain greater statistical power. Future studies should replicate our findings in different contexts with larger samples to test the robustness of our findings. In addition, this study uses a simple reflection design, that is, asking participants to write down their positions regarding the policy issues. Whether or not other alternative manipulations of reflection could, however, generate better results is still unknown. Future studies could test other forms of reflection, such as close-ended questions, or multimedia answers such as voice recording. For knowledge access, we only examined its short-term effects, while a previous study (Steenbergen et al. 2015) showed that the impact on knowledge by information materials disappeared in a long-term period. Future studies should investigate the long-term effects of intrapersonal deliberation. In addition, this study only examined a limited number of dependent variables. Future studies could also look at other political and psychological effects, such as actual behaviors in interpersonal deliberation. This would broaden our understanding of intrapersonal deliberation. In addition, this study used student samples, which lacked external validity. Future studies should examine intrapersonal deliberation pursuing more representative samples. Lastly, this study looks at a prominent and controversial issue. It is possible that for a less controversial topic, our conclusions might not hold true. For example, people might be more willing to express opinions for a less controversial issue once they gain some knowledge about the topic. Future studies should examine the generalizability of our conclusions on other issues.
In conclusion, this study could shed light on several practical and theoretical discussions. Practically, two variables in intrapersonal deliberation are tested, which could help future structured deliberation practices and studies to include these factors into the design of the deliberation process. This is especially true for online civic platforms, which should use different technological designs to implement and examine the two components of intrapersonal deliberation according to specific purposes and contexts. For example, when participants lack motivation to join a discussion, we should not overload them with too much information, which would further prevent their participation.
Theoretically, this study provides a definition of intrapersonal deliberation and examines at least two factors that could be crucial to intrapersonal deliberation. A comprehensive model that examines the mechanism and function of different components in the whole deliberation process, including both intrapersonal and interpersonal deliberation, would be necessary if we take a ‘systemic approach’ for deliberation (Mansbridge et al. 2012: 4). Research so far has overemphasized interpersonal deliberation, whereas other essential parts have long been overlooked in previous theoretical frameworks. This study shows an example to look at intrapersonal deliberation, a commonly used but often underestimated stage leading to or discouraging interpersonal deliberation. In future studies, theory building on these overlooked components of deliberation should be put into the research agenda.
Notes
- According to Price and Neijens (1997), the concept opinion quality includes stability, consistency, strength, confidence, etc. ⮭
- As the direction of attitude change might vary across different deliberation topics, we do not specify the direction of attitude change here. ⮭
- These political psychological variables were not affected by the short-term treatments, as a series of two-way ANOVAs which tested the effects of knowledge access and reflection on control variables showed no significant effects. ⮭
Funding Information
The author(s) disclose receipt of the following financial support for the research, authorship, and/or publication of this article: The research is based on a research project funded by Singapore’s Ministry of Education (MOE2013-T2-1-105).
Competing Interests
The authors have no competing interests to declare.
References
1 Aristotle. (1991). On rhetoric: A theory of civic discourse. In G. Kennedy, Trans. & Notes. Oxford: Oxford University Press.
2 Bem, D. J. (1967). Self-perception: An alternative interpretation of cognitive dissonance phenomena. Psychological Review, 74(3), 183–200. DOI: http://doi.org/10.1037/h0024835
3 Campbell, A., Gurin, G., & Miller, W. E. (1954). The voter decides. Evanston, IL: Row, Peterson.
4 Cappella, J. N., Price, V., & Nir, L. (2002). Argument repertoire as a reliable and valid measure of opinion quality: Electronic dialogue during campaign 2000. Political Communication, 19(1), 73–93. DOI: DOI: http://doi.org/10.1080/105846002317246498
5 Chambers, S. (2003). Deliberative democratic theory. Annual Review of Political Science, 6(1), 307–326. DOI: http://doi.org/10.1146/annurev.polisci.6.121901.085538
6 Chang, L., Jacobson, T. L., & Zhang, W. (2013). A communicative action approach to evaluating citizen support for a government’s smoking policies. Journal of Communication, 63(6), 1153–1174. DOI: http://doi.org/10.1111/jcom.12065
7 Cheatham, L., & Tormala, Z. L. (2015). Attitude certainty and attitudinal advocacy: The unique roles of clarity and correctness. Personality and Social Psychology Bulletin, 41(11), 1537–1550. DOI: http://doi.org/10.1177/0146167215601406
8 Cohen, J. (1997). Deliberation and democratic legitimacy. In J. Bohman & W. Rehg (Eds.), Deliberative democracy (pp. 67–91). Cambridge, MA: MIT Press.
9 Dahlberg, L. (2001). The Internet and democratic discourse: Exploring the prospects of online deliberative forums extending the public sphere. Information, Communication & Society, 4(4), 615–633. DOI: http://doi.org/10.1080/13691180110097030
10 Dahlberg, L. (2007). Rethinking the fragmentation of the cyberpublic: from consensus to contestation. New Media & Society, 9(5), 827–847. DOI: http://doi.org/10.1177/1461444807081228
11 Delli Carpini, M. X., & Keeter, S. (1996). What Americans know about politics and why it matters. New Haven, CT: Yale University Press.
12 Dobson, A. (2014). Listening for democracy: Recognition, representation, reconciliation. Oxford: Oxford University Press. DOI: http://doi.org/10.1093/acprof:oso/9780199682447.001.0001
13 Dryzek, J. S. (2000). Deliberative democracy and beyond. Oxford: Oxford University Press.
14 Dryzek, J. S. (2016). The forum, the system, and the polity three varieties of democratic theory. Political Theory, 45(5), 610–636. DOI: http://doi.org/10.1177/0090591716659114
15 Eagly, A. H., & Chaiken, S. (1995). Attitude strength, attitude structure, and resistance to change. In R. E. Petty & J. A. Kronsnick (Eds.), Attitude strength: Antecedents and consequences (pp. 413–432). Mahwah, NJ: Lawrence Erlbaum Associates.
16 Eveland, W. P., Jr., & Schmitt, J. B. (2014). Communication content and knowledge content matters: Integrating manipulation and observation in studying news and discussion learning effects. Journal of Communication, 65(1), 170–191. DOI: http://doi.org/10.1111/jcom.12138
17 Fishkin, J. S. (1995). The voice of the people: Public opinion and democracy. New Haven, CT: Yale University Press.
18 Fournier, P., Turgeon, M., Blais, A., Everitt, J., Gidengil, E., & Nevitte, N. (2011). Deliberation from within: Changing one’s mind during an interview. Political Psychology, 32(5), 885–919. DOI: http://doi.org/10.1111/j.1467-9221.2011.00835.x
19 Gastil, J. (2008). Ch. 3. Mediated deliberation and public opinion. Political Communication and Deliberation (pp. 43–76). Thousand Oaks, CA: Sage. DOI: http://doi.org/10.4135/9781483329208.n3
20 Gastil, J., Black, L., & Moscovitz, K. (2008). Ideology, attitude change, and deliberation in small face-to-face groups. Political Communication, 25(1), 23–46. DOI: http://doi.org/10.1080/10584600701807836
21 Glassman, E. L., Kim, J., Monroy-Hernández, A., & Morris, M. R. (2015). Mudslide: A spatially anchored census of student confusion for online lecture videos. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1555–1564). ACM. DOI: http://doi.org/10.1145/2702123.2702304
22 Goodin, R. E. (2000). Democratic deliberation within. Philosophy & Public Affairs, 29(1), 81–109. DOI: http://doi.org/10.1002/9780470690734.ch3
23 Goodin, R. E. (2003). Reflective democracy. Oxford: Oxford University Press. DOI: http://doi.org/10.1093/0199256179.001.0001
24 Goodin, R. E. (2008). Innovating democracy: Democratic theory and practice after the deliberative turn. Oxford: Oxford University Press. DOI: http://doi.org/10.1093/acprof:oso/9780199547944.001.0001
25 Goodin, R. E., & Niemeyer, S. J. (2003). When does deliberation begin? Internal reflection versus public discussion in deliberative democracy. Political Studies, 51(4), 627–649. DOI: http://doi.org/10.1111/j.0032-3217.2003.00450.x
26 Graber, D. A. (2001). Processing politics: Learning from television in the internet age. Chicago, IL: University of Chicago Press. DOI: http://doi.org/10.7208/chicago/9780226924762.001.0001
27 Gutmann, A., & Thompson, D. (2009). Why deliberative democracy? Princeton, NJ: Princeton University Press.
28 Habermas, J. (1984). The theory of communicative action, Vol. 1: Reason and the rationalization of society (T. McCarthy, Trans.). Boston, MA: Beacon Press.
29 Hayes, A. F., Shanahan, J., & Glynn, C. J. (2001). Willingness to express one’s opinion in a realistic situation as a function of perceived support for that opinion. International Journal of Public Opinion Research, 13(1), 45–58. DOI: http://doi.org/10.1093/ijpor/13.1.45
30 Ho, S. S., & McLeod, D. M. (2008). Social-psychological influences on opinion expression in face-to-face and computer-mediated communication. Communication Research, 35(2), 190–207. DOI: http://doi.org/10.1177/0093650207313159
31 Jung, N., Kim, Y., & de Zúniga, H. G. (2011). The mediating role of knowledge and efficacy in the effects of communication on political participation. Mass Communication and Society, 14(4), 407–430. DOI: http://doi.org/10.1080/15205436.2010.496135
32 Kim, J., Glassman, E. L., Monroy-Hernández, A., & Morris, M. R. (2015). RIMES: Embedding interactive multimedia exercises in lecture videos. In Proceedings of the 33rd Annual ACM conference on Human Factors in Computing Systems (pp. 1535–1544). ACM. DOI: http://doi.org/10.1145/2702123.2702186
33 Kim, J., Wyatt, R. O., & Katz, E. (1999). News, talk, opinion, participation: The part played by conversation in deliberative democracy. Political Communication, 16(4), 361–385. DOI: http://doi.org/10.1080/105846099198541
34 Kim, N. (2016). Beyond rationality: The role of anger and information in deliberation. Communication Research, 43(1), 3–24. DOI: http://doi.org/10.1177/0093650213510943
35 Landemore, H., & Mercier, H. (2012). Talking it out with others vs. deliberation within and the law of group polarization: Some implications of the argumentative theory of reasoning for deliberative democracy. Análise Social, 47(205), 910–934. DOI: http://doi.org/10.2139/ssrn.1660695
36 Lee, F. L. (2009). The impact of political discussion in a democratizing society: The moderating role of disagreement and support for democracy. Communication Research, 36(3), 379–399. DOI: http://doi.org/10.1177/0093650209333027
37 Luskin, R., O’Flynn, I., Fishkin, J., & Russell, D. (2014). Deliberating across deep divides. Political Studies, 62(1), 116–135. DOI: http://doi.org/10.1111/j.1467-9248.2012.01005.x
38 Mansbridge, J. J. (1983). Beyond adversary democracy. Chicago, IL: University of Chicago Press.
39 Mansbridge, J., et al. (2012). A systemic approach to deliberative democracy. In Deliberative systems: Deliberative democracy at the large scale (pp. 1–26). Cambridge, England: Cambridge University Press. DOI: http://doi.org/10.1017/CBO9781139178914.002
40 Matthes, J., Morrison, K. R., & Schemer, C. (2010). A spiral of silence for some: Attitude certainty and the expression of political minority opinions. Communication Research, 37(6), 774–800. DOI: http://doi.org/10.1177/0093650210362685
41 McBride, C. (2003). Consensus, legitimacy, and the exercise of judgement in political deliberation. Critical Review of International Social and Political Philosophy, 6(3), 104–128. DOI: http://doi.org/10.1080/1369823032000233573
42 McLeod, J. M., et al. (1999). Understanding deliberation: The effects of discussion networks on participation in a public forum. Communication Research, 26(6), 743–774. DOI: http://doi.org/10.1177/009365099026006005
43 Mendelberg, T., & Oleske, J. (2000). Race and public deliberation. Political Communication, 17(2), 169–191. DOI: http://doi.org/10.1080/105846000198468
44 Menon, S., Zhang, W., & Perrault, S. (2020). Nudge for deliberativeness: How interface features influence online discourse. In Proceedings of the SIGCHI Conference Full Papers on Human Factors in Computing Systems (CHI ’20). Hawaii: CHI ’20. DOI: http://doi.org/10.1145/3313831.3376646
45 Mercier, H., & Landemore, H. (2012). Reasoning is for arguing: Understanding the successes and failures of deliberation. Political Psychology, 33(2), 243–258. DOI: http://doi.org/10.2139/ssrn.1660695
46 Morell, M. E. (2003). Survey and experimental evidence for a reliable and valid measure of internal political efficacy. Public Opinion Quarterly, 67(4), 989–1001. DOI: http://doi.org/10.1086/378965
47 Muhlberger, P., & Weber, L. M. (2006). Lessons from the Virtual Agora Project: The effects of agency, identity, information, and deliberation on political knowledge. Journal of Public Deliberation, 2(1), 6. DOI: http://doi.org/10.16997/jdd.37
48 Mutz, D. C. (2002). The consequences of cross-cutting networks for political participation. American Journal of Political Science, 46(4), 838–855. DOI: http://doi.org/10.2307/3088437
49 Mutz, D. C. (2008). Is deliberative democracy a falsifiable theory? Annual Review of Political Science, 11(1), 521–538. DOI: http://doi.org/10.1146/annurev.polisci.11.081306.070308
50 Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. DOI: http://doi.org/10.1037/1089-2680.2.2.175
51 Perrault, S., & Zhang, W. (2019). Effects of moderation and opinion heterogeneity on attitude towards the online deliberation process. In Proceedings of the SIGCHI Conference Full Papers on Human Factors in Computing Systems (CHI ’19). Glasgow, UK. DOI: http://doi.org/10.1145/3290605.3300247
52 Petrocelli, J. V., Tormala, Z. L., & Rucker, D. D. (2007). Unpacking attitude certainty: Attitude clarity and attitude correctness. Journal of Personality and Social Psychology, 92(1), 30–41. DOI: http://doi.org/10.1037/0022-3514.92.1.30
53 Petty, R. E., & Cacioppo, J. T. (1986). Communication and persuasion: Central and peripheral routes to attitude change. New York: Springer-Verlag. DOI: http://doi.org/10.1007/978-1-4612-4964-1
54 Pingree, R. J. (2007). How messages affect their senders: A more general model of message effects and implications for deliberation. Communication Theory, 17(4), 439–461. DOI: http://doi.org/10.1111/j.1468-2885.2007.00306.x
55 Pressley, M., McDaniel, M. A., Turnure, J. E., Wood, E., & Ahmad, M. (1987). Generation and precision of elaboration: Effects on intentional and incidental learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13(2), 291–300. DOI: http://doi.org/10.1037/0278-7393.13.2.291
56 Price, V., & Neijens, P. (1997). Opinion quality in public opinion research. International Journal of Public Opinion Research, 9(4), 336–360. DOI: http://doi.org/10.1093/ijpor/9.4.336
57 Rios, K., DeMarree, K. G., & Statzer, J. (2014). Attitude certainty and conflict style divergent effects of correctness and clarity. Personality and Social Psychology Bulletin, 40(7), 819–830. DOI: http://doi.org/10.1177/0146167214528991
58 Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of Educational Research, 66(2), 181–221. DOI: http://doi.org/10.2307/1170607
59 Rucker, D. D., Petty, R. E., & Briñol, P. (2008). What’s in a frame anyway?: A meta-cognitive analysis of the impact of one versus two sided message framing on attitude certainty. Journal of Consumer Psychology, 18(2), 137–149. DOI: http://doi.org/10.1016/j.jcps.2008.01.008
60 Sanders, L. M. (1997). Against deliberation. Political Theory, 25(3), 347–376. DOI: http://doi.org/10.1177/0090591797025003002
61 Schaffer, L. C., & Hannafin, M. J. (1986). The effects of progressive interactivity on learning from interactive video. ECTJ, 34(2), 89–96.
62 Schneiderhan, E., & Khan, S. (2008). Reasons and inclusion: The foundation of deliberation. Sociological Theory, 26(1), 1–24. DOI: http://doi.org/10.1111/j.1467-9558.2008.00316.x
63 Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4(6), 592–604. DOI: http://doi.org/10.1037/0278-7393.4.6.592
64 Smith, S. M., Fabrigar, L. R., Macdougall, B. L., & Wiesenthal, N. L. (2008). The role of amount, cognitive elaboration, and structural consistency of attitude relevant knowledge in the formation of attitude certainty. European Journal of Social Psychology, 38(2), 280–295. DOI: http://doi.org/10.1002/ejsp.447
65 Steenbergen, M., Bachtiger, A., Pedrini, S., & Gautschi, T. (2015). Information, deliberation, and direct democracy: Evidence from the Swiss Expulsion Initiative. In S. Coleman, A. Przybylska, & Y. Sintomer (Eds.), Deliberation and democracy: Innovative processes and institutions (pp. 187–203). Frankfurt, Germany: Peter Lang.
66 Stromer-Galley, J., & Muhlberger, P. (2009). Agreement and disagreement in group deliberation: Effects on deliberation satisfaction, future engagement, and decision legitimacy. Political Communication, 26(2), 173–192. DOI: http://doi.org/10.1080/10584600902850775
67 Sullivan, J. L., & Transue, J. E. (1999). The psychological underpinnings of democracy: A selective review of research on political tolerance, interpersonal trust, and social capital. Annual Review of Psychology, 50(1), 625–650. DOI: http://doi.org/10.1146/annurev.psych.50.1.625
68 Tocqueville, A. (1996). Democracy in America. In M. Cohen & N. Fermon (Eds.), Princeton readings in political thought (pp. 398–424). Princeton, NJ: Princeton University Press. (Original work published 1835)
69 Tormala, Z. L., & Rucker, D. D. (2007). Attitude certainty: A review of past findings and emerging perspectives. Social and Personality Psychology Compass, 1(1), 469–492. DOI: http://doi.org/10.1111/j.1751-9004.2007.00025.x
70 Weinmann, C. (2016). Measuring political thinking: Development and validation of a scale for ‘deliberation within’. Paper presented at the 66th Annual Conference of the International Communication Association (ICA), Fukuoka, Japan.
71 Weinmann, C. (2018). Measuring political thinking: Development and validation of a scale for ‘deliberation within’. Political Psychology, 39(2), 365–380. DOI: http://doi.org/10.1111/pops.12423
72 Weinmann, C., & Vorderer, P. (2015). Deliberative democracy and entertainment: Bridging two separate theoretical traditions. Paper presented at the 65th Annual Conference of the International Communication Association (ICA), San Juan, Puerto Rico.
73 Xiao, L., et al. (2015). Design for online deliberative processes and technologies: Towards a multidisciplinary research agenda. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 865–868). Seoul, South Korea. DOI: http://doi.org/10.1145/2702613.2727687
74 Young, I. M. (1996). Communication and the other: Beyond deliberative democracy. In S. Benhabib (Ed.), Democracy and difference. Contesting the boundaries of the political (pp. 120–135). Princeton, NJ: Princeton University Press.
75 Young, I. M. (2002). Inclusion and democracy. Oxford: Oxford University Press.
76 Zhang, W. (2015). Perceived procedural fairness in deliberation: Predictors and effects. Communication Research, 42(3), 345–364. DOI: http://doi.org/10.1177/0093650212469544
77 Zhang, W. (2016). Social media and elections in Singapore: Comparing 2011 and 2015. Chinese Journal of Communication, 9(4), 367–384. DOI: http://doi.org/10.1080/17544750.2016.1231129
78 Zhang, W., & Chang, L. (2014). Perceived speech conditions and disagreement of everyday talk: A proceduralist perspective of citizen deliberation. Communication Theory, 24(2), 124–145. DOI: http://doi.org/10.1111/comt.12034