Introduction
Deliberative mini-publics have garnered significant attention as a practice of deliberative democracy (Bächtiger and Parkinson, 2019). Mini-publics are a method of forming a microcosm of citizens by selecting participants through random sampling and utilizing the outcomes of their deliberation for policy formation and political decision-making (Smith, 2009; Smith and Setälä, 2018). Amongst various mini-publics methodologies, deliberative polls, which aim to form ‘refined’ rather than ‘raw’ public opinion through deliberation, has been implemented in various countries and regions (Deliberative Democracy Lab, n.d.; Fishkin, 2009; He et al., 2021).
This study examines the correlation between opinion changes and deliberation quality in deliberative polls. Some scholars in deliberative democracy emphasize the alteration of participants’ preferences through rational discourse. It has been argued that a central tenet of all deliberative theories is that deliberation has the transformative power to shape opinions and preferences (Chambers, 2003; Suiter et al., 2016). Of course, attitude change is only one of the outcomes that are theoretically and empirically expected from deliberative processes. Desirable outcomes anticipated from deliberation encompass a wide spectrum, including increased political tolerance, a stronger sense of political efficacy, and the cultivation of social trust (Mutz 2008). Nonetheless, especially in deliberative polls, which measure citizens’ opinions in pre- and post-deliberation questionnaires, it is expected that citizens transform their preferences and attitudes by exchanging arguments with other participants; Fishkin emphasized the change in policy attitudes as the ‘punchline of a deliberative poll’ (2009, p134).
However, the relation between attitude changes and the quality of deliberation is not straightforward. Prior empirical studies show inconclusive and mixed results on whether a high quality of deliberation leads to opinion changes. As discussed later, while some studies have demonstrated that exchange of reasoned arguments induces change in attitudes (Gerber et al., 2018; Himmelroos and Christensen, 2014), others have found that the quality of deliberation, as either reasoning or reflection, does not consistently impact attitude changes (Caluwaerts and Reuchamps, 2014; Gerber et al., 2014; D. Sanders, 2012). It has been also suggested that opinion changes occur due to social pressure and group polarization rather than knowledge gain and reflection through exchanging reasoned arguments (L. M. Sanders, 1997; Sunstein, 2000). To elucidate whether opinion change is truly desirable for democratic deliberation, it is significant to examine if it is related to the quality of deliberation.
On the other hand, while reflection induced through the exchange of reasoned arguments has been advocated by the Habermasian communicative theory, alternative forms of communication such as storytelling, greetings, and rhetoric have received increasing attention as significant components of deliberation (Dryzek, 2000; Polletta and Gardner, 2018; Young, 2000). Some studies have examined the effects of alternative forms of communication on deliberation (Black, 2008, 2009; Jaramillo and Steiner, 2014; Maia et al., 2020; Muradova, 2021; Polletta and Lee, 2006; Ryfe, 2006), but their impacts on attitude transformation have not yet been fully elucidated.
It is significant, therefore, to connect broad variations in the quality of deliberation with the likelihood of opinion changes among participants (Knobloch and Gastil, 2022). This study examined this connection, focusing on the effects of reasoned arguments and storytelling on transforming participants’ attitudes. Drawing on the data of a deliberation event modeled after a deliberative poll that was held in 2019 on the topic of resumption of nuclear power plants and local agreement in Japan, we evaluated the qualities of discussion with an index according to the Discourse Quality Index (Steenbergen et al., 2003; Steiner et al., 2004) and analyzed their relationship to participants’ attitudes transformation. While deliberative mini-publics have been widely practiced, little is known about how citizens communicate in such forums and how this affects their outcomes (Black, 2013). By examining the two forms of communication and their associations with changes in participants’ opinions, this study extends empirical understanding of deliberative democracy.
The remainder of this paper is organized as follows: In the next section, we briefly review the literature on deliberations and opinion changes. We then present our data, methods, results, and subsequent discussions. Finally, we summarize the key findings and their implications.
Deliberation and opinion changes
Deliberative scholars have extensively researched opinion transformation through deliberation and its contributing factors (Jacquet and van der Does, 2021). Prior studies have explored the effect of demographic factors such as age, gender, class, educational background, and ethnicity on opinion changes through deliberations (Luskin et al., 2002; Muhlberger, 2006; Suiter et al., 2016). For example, based on data from a nationwide exercise in deliberative democracy in Ireland, Suiter et al. (2016) found that participants most likely to shift their opinions following deliberation were under 65 years old and had median levels of knowledge. Other studies have aimed to discern whether information acquisition (internal reflection with knowledge gain) or deliberation affects opinion changes (Goodin and Niemeyer, 2003; Luskin et al., 2002; O’Malley et al., 2020; Tanaka, 2018). While analyzing the deliberations of a citizen’s jury on an Australian environmental issue, Goodin and Niemeyer (2003) found that jurors’ attitudes changed more in response to the ‘information’ phase of the jury proceedings, involving a large degree of ‘deliberation within,’ than during the formal ‘discussion’ phase. Additionally, Knobloch and Gastil (2022), drawing on the data of 15 Citizen’s Initiative Reviews held from 2010 to 2018, found that the duration and official authorization of the processes yielded a greater frequency of changing one’s opinion.
However, it has been understudied if there indeed is a link between opinion changes and the quality of deliberation (Grönlund et al., 2014). Few studies have attempted to examine the relationship between the quality of deliberation and opinion changes. Most of these studies aim to empirically test the core expectation of deliberative democracy, wherein individuals change their preferences and attitudes as a result of the reflection induced through the exchange of reasoned arguments (Gerber et al., 2014, 2018; Himmelroos & Christensen, 2014; Neblo, 2007; D. Sanders, 2012). Using data from a mini-public concerning the use of nuclear power in Finland, Himmelroos and Christensen (2014) concluded that deliberative reasoning, evaluated using the Discourse Quality Index, and issue awareness were significant predictors of opinion changes. Similarly, Gerber et al. (2018), drawing on EuroPolis data, showed that participants often changed their opinions when rational justification was used in discussions. However, using data from EuroPolis, D. Sanders (2012) tested five different explanations of deliberators’ opinion changes, including discussion quality subjectively evaluated by a survey. He reported that none of them was satisfactory. Gerber et al. (2014) explored which affected opinions: statements backed by reason (deliberative persuasion) or the most frequently repeated position (non-deliberative persuasion), concluding that persuasion-influenced opinion change differed by topic. Caluwaerts and Reuchamps (2014) found that the quality of deliberation had only a limited impact upon attitude changes, casting doubt on the transformative effects of deliberation. These conflicting results indicate that it is unclear whether exchanging arguments supported by reasons is indeed a substantial cause of opinion change.
Furthermore, prior research has not clarified the quality of deliberation that influences attitude transformation (Polletta and Gardner, 2018). The Discourse Quality Index (Steenbergen et al., 2003; Steiner et al., 2004), based on the Habermasian normative theory of communication, is often used to evaluate the quality of deliberation (Gerber et al., 2014, 2018; Himmelroos and Christensen, 2014; Lindell, 2015; Westwood, 2015). However, when measuring the quality of discussion and examining its association with opinion changes, the prior studies lumped together the main items of the Discourse Quality Index (DQI), such as ‘Level of justification,’ ‘Content of justification,’ and ‘Respect,’ into a single variable; it is not clear what kind of quality of deliberation affects preference changes.
Besides, the significance of various communication modes has been recognized. Among such alternative communications, storytelling and personal testimonies or narratives have attracted special attention (Black, 2008, 2009, 2013; Dryzek, 2000; McCoy and Scully, 2002; Polletta and Gardner, 2018; Young, 2000). Indeed, ordinary citizens often use narratives and tell stories, especially in small-group discussions (Maia et al., 2020). Young (2000) argues that narrative is one of the three modes of communication used to mitigate internal exclusion by empowering relatively disfranchised groups and providing a means to engage in fruitful debate for those who have different experiences and beliefs and do not share enough premises. From an empirical study with mock jury deliberations, Feller (2018) contends that people of perceived lower social standing have a greater desire to bolster their opinions with citations of personal experience. Dryzek (2000) admits the importance of alternative modes of communication and incorporates them into his theory of discursive democracy, as long as they are non-coercive and capable of connecting the particular to the general. The DQI also incorporates storytelling as a type II standard (Bächtiger et al., 2009). Muradova et al. (2020) include storytelling in the operationalization of effective communication to test whether it contributes to explaining outcomes of deliberation on climate change.
It is true that potential drawbacks of storytelling in deliberative process have also been acknowledged. For example, Dryzek (2000) argued that storytelling could be coercive when group norms limit the range of acceptable stories. Similarly, Black (2009) highlighted the potential of storytelling to divide group members into factions, create a sense of false consensus by overemphasizing collective identities, and cover real differences among group members. However, prior research has empirically examined the importance of personal storytelling in deliberative processes, highlighting that it could lower structural, psychological, and social barriers to discussion (Ryfe, 2006). Additionally, introductory stories have been shown to engage participants by connecting abstract issues with their lived experiences and helping establish the legitimacy of individuals’ experiences in the discussion (Black, 2009). Furthermore, storytelling has been found to help manage differences and disagreements among participants (Polletta and Lee, 2006; Ryfe, 2006), encourage perspective-taking interplaying with fact-based argumentation (Muradova, 2021), and elicit responses in online discussions (Polletta and Lee, 2006). It has been argued that personal stories have the potential to transform discussions from low- to high-level deliberation (Jaramillo and Steiner, 2014) and that both storytelling and reason-giving are important components in good deliberation moments (Maia et al., 2020). Therefore, when examining the quality of deliberation concerning opinion changes, it is crucial to consider alternative forms of communication, such as personal storytelling.
It should be noted that we do not consider storytelling and reason-giving to be mutually exclusive. Karpowitz and Raphael (2014, p219) defines reason-giving as ‘any statement that answers “the why question” about the basis for one’s position,’ encompassing not only rational argumentation but also affective reasoning based on emotion. Maia and Hauber (2020) empirically demonstrated that emotion is a central component in the reasoning process. Alternative forms of communication such as storytelling can provide compelling reasons to support an argument (Adams, 2014), by formulating one’s opinions and craft justifications for those opinions (Polletta and Lee, 2006). Indeed, in deliberation, personal narratives are used frequently as a basis or reference to support positions (Dutwin, 2001; Stromer-galley, 2007). Therefore, reason-giving extends beyond mere rational argumentation with fact-based evidence and coherent logic; storytelling is an essential part of argumentation (Dutwin, 2001). As Gerber et al. (2018) argues, ‘the classic distinction between rational discourse and alternative forms of communication may be misleading, since high-skilled deliberators also use personal experiences to back up their positions and arguments’ (Gerber et al., 2018, p1107). It is important to note that not all storytelling serves as reason-giving. As previously discussed, storytelling serves a variety of functions in a deliberative discussion. As Black (2009) summarizes, ‘stories can advance arguments, promote compromise, build community, invite dialogue, manage conflict, and help group members to identify and clarify their own perspectives’ (Black, 2009, p4). Thus, in this study, we assume that storytelling can influence opinion changes through its multifaceted roles and functions, including reason-giving.
Our study delves into the deliberative quality that affects opinion changes, not only focusing on reason-giving but also on narratives of personal experiences. As discussed above, the relationship between the quality of deliberation and opinion changes is intricate, and existing studies yield mixed and inconclusive results. Furthermore, prior research tends to emphasize reflection through reasoning as the quality of deliberation and have overlooked the influence of alternative communication such as storytelling on attitude changes. Drawing on the data of a deliberative event modeled after a deliberative poll, this study advances our understanding of the connection between the quality of deliberation and opinion transformation, contributing to the development of empirical research on deliberative democracy.
Methods
Deliberation event
The data were obtained from a mini-public deliberation event that mimicked a deliberative poll held in 2019 in Shizuoka Prefecture, Japan. The event focused on restarting the Hamaoka nuclear power plant and the issue of local agreement. Following the Fukushima nuclear accident in March 2011, Japan has grappled with whether to restart nuclear power plants and the process of obtaining a local agreement for such a decision (Nakazawa and Tatsumi, 2022). During the event, the participants were asked to discuss how local agreements should be made regarding the resumption of the nuclear power plant.
The deliberation event was conducted face-to-face and online. The first questionnaire survey (T1) was administered to a random sample of 5,000 individuals from the electoral roll of 35 municipalities in Shizuoka Prefecture,1 resulting in 2,052 valid responses. A total of 339 respondents who expressed willingness to participate in the deliberation event were sent information material on the topic and an invitation letter. Those who preferred to participate in the face-to-face event at T1 were invited to participate in the face-to-face event, whereas those who preferred the online event were invited to participate in the online event. To ensure that sociodemographic characteristics were balanced between the two events, the researchers assigned participants willing to participate face-to-face or online and sent an appropriate invitation. Ultimately, 65 individuals participated in face-to-face deliberations, and 51 participated in online deliberations.
This study used data from face-to-face deliberations. In the face-to-face event, the second questionnaire survey (T2) was administered to participants before a 90-minute discussion in eight small groups of 6–9 members, during which participants were required to deliberate and decide on questions to pose to a panel of experts, comprising four specialists in nuclear power and decision-making. After receiving answers from the expert panel, the participants engaged in another 90-minute discussion in their small groups, followed by the third questionnaire survey (T3). A trained facilitator conducted each group discussion. Face-to-face deliberations by eight small groups were recorded and transcribed into 4,178 statements (excluding facilitator comments). These statements and answers to the three questionnaire surveys (T1-3) were used as data to evaluate the quality of deliberation and attitude changes of the participants.2
Quality of deliberation
The statements were coded according to an index established based on the Discourse Quality Index (DQI). The DQI, which is an index used to evaluate the quality of deliberation grounded on the Habermasian logic of communicative action, includes items such as ‘Level of justification,’ ‘Content of justification,’ ‘Respect,’ and ‘Constructive politics’ (Steenbergen et al., 2003; Steiner et al., 2004). Subsequently, Bächtiger et al. (2009) expanded the index to include items such as storytelling and deliberative negotiation, which measure alternative forms of communication. The DQI has been used to evaluate the quality of small group discussions, including deliberative mini-publics, deliberative abilities, and equality among participants (Gerber et al., 2014, 2018; Himmelroos, 2017; Himmelroos and Christensen, 2014).
However, the original DQI is not directly applicable to our discussion. The DQI was originally developed to study debates in parliament (Steiner et al., 2004) and has limitations when applied to small group discussions, which are often characterized by quick gives-and-takes with many shortcuts (Jaramillo and Steiner, 2014). Additionally, short back-channel comments (referred to as ‘aiduchi’ in Japanese) were counted as one statement in our data; a remark was often divided into several statements by back-channel comments, which made it difficult for a statement to include both arguments and reasons (Tanaka, 2018). Similarly, Souma et al. (2022) argued that the original DQI is problematic when applied to small-group discussions in Japan, pointing out its assumptions that discussions are dichotomous and adversarial and that participants have clear opinions and reasons. It has also been noted that the DQI lacks clear ‘threshold values’ to determine whether deliberation is of sufficient quality (Bächtiger et al., 2009).
To address these issues, we modified the index to make it more suitable for our discussing data and research objectives. We separated ‘Opinion’ and ‘Reasons’ as independent items to accommodate the tendency for a statement to be interrupted and shortened. Furthermore, a statement was coded for each item with a binary value of ‘0’ or ‘1,’ with ‘1’ assigned if the statement fit the item and ‘0’ if it did not. The original DQI rates the degree of each item on a scale of 3 to 4. However, since it is extremely difficult to determine the degree of each criterion in small group discussions by ordinary citizens, we adopted a binary evaluation of ‘0’ or ‘1’ following Souma et al. (2022) and Tanaka (2018).
Four coders, who were uninformed of the study’s purpose, coded the statements after the training. Two coders were assigned to each of the eight small discussion groups. From the 30 items established for evaluation,3 we utilized three for our analysis: ‘Opinions,’ ‘Reasons,’ and ‘Personal experiences’ (Table 1). ‘Opinions’ indicates if a speaker’s opinion on the discussion topic is expressed, while ‘Reasons’ shows if justifications supporting that opinion are provided.4 We selected ‘Opinions’ and ‘Reasons’ to represent the degree of reasoned arguments. As ‘Personal experiences’ denotes a statement that includes a participant’s own experiences relevant to the topic, we selected it as a representative indicator of storytelling as an alternative form of communication.
Items used to evaluate the quality of deliberation.
| Items | Coding criteria |
| Opinions | Stating their opinion relevant to the topic |
| Reasons | Giving reasons for their opinion |
| Personal experiences | Including a participant’s own experiences relevant to the topic |
Note that each DQI item is not mutually exclusive: a statement may contain ‘Opinions,’ ‘Reasons,’ and ‘Personal experiences’ simultaneously. A single statement could include ‘Reasons’ for one aspect while containing ‘Personal experiences’ for another part. As discussed earlier, storytelling can serve as a means of providing a reason; therefore, if a speaker’s personal experience forms the basis of their opinion, the statement is considered under ‘Reasons.’ Consequently, the degree of reasoned argument measured by ‘Opinions’ and ‘Reasons’ does not solely indicate purely rational discourse; rather, it denotes the expression of a speaker’s position with any form of support for that position, including storytelling. On the other hand, ‘Personal experiences’ was used to specifically indicate the extent of narrating personal stories, which we assume serve diverse roles and functions, including reason-giving, in the deliberation.
Following previous studies, we treated the amended DQI scores as group variables (Gerber et al., 2014; Neblo, 2007). Using a union counting method (logical OR), a statement was counted as ‘1’ if either one of the two coders had coded it as ‘1.’ Note that, for each group, the scores were calculated by dividing the number of statements corresponding to each item by the total number of statements; the scores indicate the proportion of each DQI item in the total number of statements in each group.5
Attitudes changes
We analyzed the results of three questionnaire surveys (T1–T3) regarding attitude changes. Specifically, we examined changes in participants’ attitudes towards the local agreement process for the resumption of the Hamaoka nuclear power plant. In the questionnaires, respondents were asked to indicate, on a five-point scale (‘1’ indicating ‘agree’ and ‘5’ signifying ‘disagree’), their level of agreement or disagreement with each of the following statements:
| S1. | The restart of NPPs should be judged by the areas where direct damage such as pollution and evacuation is expected. |
| S2. | The judgment of the areas where not just direct damage but reputational damage is expected should be respected in the restart of NPPs. |
| S3. | The judgment of the areas where damage to industries and employment and a decrease of subsidies are expected if the Hamaoka NPP would not restart should be respected more. |
| S4. | The judgment of Omaezaki city,6 which has accepted the nuclear power plants, should be respected in the restart of the Hamaoka NPP. |
| S5. | Urban areas that consume electricity from NPPs have the right and responsibility to decide on the restart of the Hamaoka NPP as well. |
| S6. | Once the safety examination by Nuclear Regulation Authority is completed, power companies should be able to restart NPPs without agreement from local autonomies. |
| S7. | As the restart of NPPs is a national issue, the Japanese government should be responsible and make the judgment regarding the restart. |
| S8. | The local assemblies and governor/mayors who are elected should be responsible and make the judgement regarding the restart. |
| S9. | As the problems related to the restart of NPPs are too difficult for ordinary citizens to understand, the judgment should be left to experts. |
| S10. | The restart of NPPs should be judged by the will of ordinary citizens without leaving it to politicians and experts. |
| S11. | The governments, companies, and experts who are promoting the restart of NPPs provide accurate information to the people. |
S1–5 pertained to which geographic areas should be heard and respected in the local agreement process, while S6–11 concerned which actors’ views should be heard and respected (Nakazawa and Tatsumi, 2022). Opinion change was calculated as the difference in attitudes toward these statements between T2 (before deliberation) and T3 (after deliberation).
To consider the possibility that different forms of deliberation quality can affect different attitudes (Polletta and Lee, 2006) and to maintain simplicity in our analysis model simultaneously, we calculated subscale scores based on the results of an exploratory factor analysis of T1 data (Tatsumi and Nakazawa, 2021).7 Factor analysis revealed three factors underlying respondents’ attitudes toward local agreements: ‘conventional decision-makers’ (F1), ‘narrow localism’ (F2), and ‘national interests’ (F3). Table 2 lists the statements corresponding to each factor.
Subscale scores and corresponding factors.
| Subscale scores | Factors | Statements |
| SS-I | F1. conventional decision-makers | S6, S2*, S9, S10*, S11, S8 |
| SS-II | F2. narrow localism | S1, S4, S3 |
| SS-III | F3. national interests | S5, S7 |
Note: * indicates a statement for which the direction of the five-point scale was reversed.
The subscale scores were calculated by converting positive attitudes (agree) to 2 points, neutral attitudes (neither agree nor disagree) to 0 points, and negative attitudes (disagree) to –2 points for the statements evaluated on a five-point Likert scale. A ‘0’ score indicates a neutral attitude, and higher scores indicate a more positive attitude. The average score of statements comprising each factor was used as the corresponding subscale. For each subscale score, individual attitude changes were calculated as the absolute value of the difference between the T2 and T3 scores without considering the direction of approval or disapproval. It would be also possible to take into account the direction of opinion change in the analysis (Himmelroos and Christensen, 2014). However, since one of the purposes of this study is to examine the desirability of opinion change as an indicator of democratic deliberation, the absolute amount of opinion change was used as the dependent variable.
Analysis
First, we provided descriptive statistics of the modified DQI scores and attitude changes. We then utilized multiple regression analysis to investigate the factors contributing to individual attitude change in each F1–3, with the change in each subscale score as the dependent variable. The independent variables included the scores of the three items from the amended DQI: ‘Opinions,’ ‘Reasons,’ and ‘Personal experiences.’ ‘Opinions’ and ‘Reasons’ are indicators of the quality of reason-giving, while ‘Personal experiences’ is incorporated in our model as an indicator of alternative forms of communication.
We also incorporated control variables in our model: the number of statements made by individual participants and the group and sociodemographic factors. Prior studies have illustrated the influence of demographic factors on opinion changes through deliberations (Luskin et al., 2002; Muhlberger, 2006; Suiter et al., 2016). As sociodemographic variables, age, educational background, gender (dummy variable with female as ‘1’), and residence were incorporated in our model. The educational background was treated as an interval scale, with ‘1’ indicating primary education (elementary and junior high school), ‘2’ indicating secondary education (high school), ‘3’ indicating higher education (junior college or technical college), ‘4’ indicating higher education (university, master’s degree or higher).8 Residence was incorporated into our model to control for the potential socioeconomic impacts of the nuclear power plant and energy supply on participants’ attitudes toward the issue and engagement in the discussion (Nakazawa and Tatsumi, 2022). Residence was categorized into four areas: areas with evacuation plans in preparation for a nuclear accident (i.e., Urgent Protective action planning Zone, or UPZ: areas within 31 km of the Hamaoka nuclear power plant) and central, western, and eastern areas outside of the UPZ. It is worth noting that the eastern area is supplied by a different power company and is not directly supplied by the Hamaoka nuclear power plant.
In our multiple regression analysis, we presented the results of three models. Model 1 consisted solely of the scores of three items of DQI and the number of statements as independent variables. Model 2 incorporated these variables along with age, education history, and gender. Model 3 was created with all variables, including place of residence. The regression model was a linear function, the variables were standardized, and the coefficients were estimated using the least-squares method. Given the relatively small sample size and the exploratory nature of the current study, the results were evaluated using a significance level of 10%.
Results
Quality of deliberation
Table 3 shows the results of the discourse quality coding by group. A total of 3,429 statements, out of 4,187, were classified as ‘Opinions,’ while 1,042 statements were categorized as ‘Reasons.’ In contrast, the number of statements coded as ‘Personal experiences’ was relatively small, with only 99 statements falling under this category. The group scores of these items exhibited some variation, ranging from 0.72 to 0.89 for ‘Opinions,’ 0.13 to 0.39 for ‘Reasons,’ and 0.01 to 0.05 for ‘Personal experiences.’
Group scores of the amended DQI and number of statements.
| DQI items | G1 | G2 | G3 | G4 | G5 | G6 | G7 | G8 | All | Kappa Coefficient | Gwet’s AC1 |
| Opinions | 0.72 (496) |
0.89 (474) |
0.84 (263) |
0.83 (657) |
0.81 (695) |
0.88 (238) |
0.83 (299) |
0.80 (307) |
0.82 (3429) |
0.52 | 0.68 |
| Reasons | 0.21 (147) |
0.29 (153) |
0.28 (89) |
0.27 (210) |
0.13 (110) |
0.39 (105) |
0.32 (114) |
0.30 (114) |
0.25 (1042) |
0.64 | 0.84 |
| Personal experiences | 0.02 (15) |
0.03 (16) |
0.03 (10) |
0.01 (10) |
0.02 (13) |
0.05 (14) |
0.03 (12) |
0.02 (9) |
0.02 (99) |
0.57 | 0.99 |
| # of statements in the group | 689 | 530 | 314 | 787 | 854 | 271 | 359 | 383 | 4187 |
Note: The numbers of statements corresponding to each DQI item are shown in parentheses. DQI scores were calculated by dividing the number of statements corresponding to each item by the total number of statements in the group.
Reproducibility between the two coders was relatively good. Cohen’s kappa coefficient was 0.69, while Gwet’s AC1 was 0.93 for the 30 items. Note that the kappa coefficients of ‘Opinions’ and ‘Personal experiences’ slightly did not reach > 0.58, which is regarded as ‘good’ according to Pinto’s criteria (Pinto et al., 1988). However, especially when the proportion of relevant items is extremely low, it is encouraged to use Gwet’s AC1 alongside Cohen’s Kappa (Wongpakaran et al., 2013). Therefore, we used these three parameters in the present study.
Statements coded as ‘Personal experiences’ included experiences with nuclear power plants and various related events and political participation. For example, some participants referred to their personal experiences with nuclear power plants when explaining why they had decided to participate in the deliberative event. Participants talked about their visits to the Hamaoka nuclear power plant, including the visitor center. They shared their experiences of the power outage after the Great East Japan Earthquake in March 2011 and the typhoon in 2018. Another notable example is their experience of political participation, such as the referendum held in Hamamatsu City in 2019 and citizens’ meetings held by local governments.
Attitudes changes
Table 4 illustrates the magnitude of change in opinion, represented as the absolute value of the difference between the participants’ subscale scores before and after the discussion.9 The mean change in opinion for the SS-I was smaller than that for the other two subscales, with a mean change of 0.41 (std = 0.31). Changes in SS-II and SS-III showed a weak positive correlation (Table 5).
Magnitude of opinion change by group.
| G1 | G2 | G3 | G4 | G5 | G6 | G7 | G8 | Total | |
| # of respondents | 8 | 8 | 6 | 8 | 8 | 9 | 9 | 9 | 65 |
| SS-I | 0.58 (0.47) | 0.53 (0.22) | 0.31 (0.19) | 0.38 (0.34) | 0.52 (0.27) | 0.41 (0.31) | 0.39 (0.18) | 0.22 (0.22) | 0.41 (0.31) |
| SS-II | 0.71 (0.42) | 0.56 (0.46) | 0.86 (0.92) | 1.00 (0.94) | 0.83 (0.60) | 0.44 (0.68) | 0.37 (0.48) | 0.78 (0.52) | 0.69 (0.69) |
| SS-III | 0.69 (0.50) | 0.58 (0.34) | 0.71 (0.65) | 0.57 (0.62) | 0.81 (1.06) | 0.22 (0.42) | 0.56 (0.60) | 1.22 (0.79) | 0.67 (0.72) |
Note: Standard deviation is shown in parentheses.
Correlation coefficients and partial correlation coefficients between changes in subscale scores.
| SS-I | SS-II | SS-III | |
| SS-I | 0.08 | 0.05 | |
| SS-II | 0.07 | 0.32 | |
| SS-III | 0.03 | 0.32 |
Note: The upper-right panel shows the correlation coefficients, and the lower left panel shows the partial correlation coefficients.
Multiple regression analysis
The descriptive statistics of the explanatory variables are presented in Table 6. Table 7 presents the correlation and partial correlation coefficients of the explanatory variables. The number of statements in the group showed a relatively strong inverse correlation with the scores of ‘Reasons’ and ‘Personal experiences.’ However, no multicollinearity was detected among the explanatory variables.
Descriptive statistics for explanatory variables (all of the participants).
| Mean | SD | # of respondents | |
| # of statements | 67.54 | 63.02 | 65 |
| # of statements in group | 514.60 | 216.13 | 65 |
| DQI (Opinions) | 0.825 | 0.049 | 65 |
| DQI (Reasons) | 0.275 | 0.073 | 65 |
| DQI (Personal Experiences) | 0.028 | 0.012 | 65 |
| Age | 56.32 | 15.55 | 65 |
| Education | 2.73 | 1.00 | 64 |
| Gender (male: ref) | 35 | ||
| Gender (female) | 30 | ||
| Area (midland: ref) | 15 | ||
| Area (UPZ) | 17 | ||
| Area (west) | 14 | ||
| Area (east) | 19 |
Note: Nonresponses were excluded from the analysis. ‘Mean’ shows average values for all groups.
Correlation coefficients and partial correlation coefficients of explanatory variables.
| # of statements | # of statements in group |
DQI (Opinions) |
DQI (Reasons) |
DQI (Personal Experiences) |
Age | Education | |
| # of statements | 0.52 | –0.10 | –0.37 | –0.43 | 0.08 | 0.19 | |
| # of statements in group | 0.34 | –0.39 | –0.84 | –0.83 | –0.10 | –0.12 | |
| DQI (Opinions) | 0.08 | 0.19 | 0.53 | 0.50 | –0.05 | 0.04 | |
| DQI (Reasons) | 0.15 | –0.58 | 0.30 | 0.79 | –0.23 | –0.21 | |
| DQI (Personal Experiences) | –0.11 | –0.41 | 0.25 | 0.27 | 0.25 | 0.20 | |
| Age | –0.10 | –0.11 | –0.04 | 0.02 | 0.16 | –0.44 | |
| Education | 0.11 | –0.05 | 0.09 | 0.02 | 0.08 | –0.49 |
Note: The upper right panel shows the correlation coefficients, and the lower left panel shows the partial correlation coefficients.
Tables 8, 9, 10 provide a summary of the results from the multiple regression analysis.10 Regarding the model fitness assessed through AIC (Akaike Information Criterion), Model 2, which incorporated sociodemographic variables excluding residential area, demonstrated the highest fitness for SS-Ⅰ and SS-Ⅱ. For SS-Ⅲ, while Model 1 exhibited the smallest AIC, the difference was not significant. On the other hand, the coefficient of determination was highest in Model 3 across all sub-scores. Overall, the model fitness is higher in SS-Ⅰ. Additionally, while the statistical power (1–β)11 is sufficiently high for SS-I in Model 2 and 3, it is low for SS-Ⅱ and SS-Ⅲ in all three models.
Results of regression analysis: SS-I (F1. conventional decision-makers).
| Model 1 | Model 2 | Model 3 | ||||
| Estimate | Pr(sign) | Estimate | Pr(sign) | Estimate | Pr(sign) | |
| Intercept | 0.41 | ** | 0.40 | ** | 0.40 | ** |
| # of statements | 0.04 | n.s. | 0.00 | n.s. | 0.00 | n.s. |
| # of statements in group | 0.12 | n.s. | 0.11 | n.s. | 0.10 | n.s. |
| DQI (Opinions) | –0.03 | n.s. | –0.04 | n.s. | –0.03 | n.s. |
| DQI (Reasons) | –0.06 | n.s. | –0.13 | n.s. | –0.15 | † |
| DQI (Personal experiences) | 0.16 | * | 0.24 | ** | 0.25 | ** |
| Age | –0.06 | n.s. | –0.06 | n.s. | ||
| Education | 0.02 | n.s. | 0.02 | n.s. | ||
| Gender (female = 1) | –0.12 | * | –0.13 | ** | ||
| Area (UPZ = 1) | –0.04 | n.s. | ||||
| Area (west=1) | 0.00 | n.s. | ||||
| Area (east=1) | 0.04 | n.s. | ||||
| Sample Size | 63 | 62 | 62 | |||
| AIC | 38.47 | 32.23 | 35.10 | |||
| R2 | 0.12 | 0.24 | 0.28 | |||
| Power(1–β) | 0.55 | 0.86 | 0.88 | |||
**p < 0.01, *p < 0.05, †p < 0.10.
Results of regression analysis: SS-II (F2. narrow localism).
| Model 1 | Model 2 | Model 3 | ||||
| Estimate | Pr(sign) | Estimate | Pr(sign) | Estimate | Pr(sign) | |
| Intercept | 0.69 | ** | 0.66 | ** | 0.65 | ** |
| # of statements | –0.05 | n.s. | –0.04 | n.s. | –0.05 | n.s. |
| # of statements in group | 0.01 | n.s. | –0.07 | n.s. | –0.10 | n.s. |
| DQI (Opinions) | 0.03 | n.s. | 0.03 | n.s. | 0.05 | n.s. |
| DQI (Reasons) | 0.02 | n.s. | –0.04 | n.s. | –0.09 | n.s. |
| DQI (Personal experiences) | –0.20 | n.s. | –0.25 | n.s. | –0.23 | n.s. |
| Age | –0.05 | n.s. | –0.04 | n.s. | ||
| Education | –0.05 | n.s. | –0.04 | n.s. | ||
| Gender (female = 1) | –0.02 | n.s. | –0.04 | n.s. | ||
| Area (UPZ = 1) | –0.02 | n.s. | ||||
| Area (west = 1) | 0.07 | n.s. | ||||
| Area (east = 1) | 0.11 | n.s. | ||||
| Sample Size | 64 | 63 | 63 | |||
| AIC | 143.61 | 141.12 | 144.85 | |||
| R2 | 0.06 | 0.11 | 0.14 | |||
| Power(1–β) | 0.27 | 0.40 | 0.44 | |||
**p < 0.01, *p < 0.05, †p < 0.10.
Results of regression analysis: SS-III (F3. national interest).
| Model 1 | Model 2 | Model 3 | ||||
| Estimate | Pr(sign) | Estimate | Pr(sign) | Estimate | Pr(sign) | |
| Intercept | 0.68 | ** | 0.66 | ** | 0.66 | ** |
| # of statements | –0.08 | n.s. | –0.08 | n.s. | –0.08 | n.s. |
| # of statements in group | –0.32 | n.s. | –0.34 | n.s. | –0.33 | n.s. |
| DQI (Opinions) | –0.01 | n.s. | –0.01 | n.s. | –0.02 | n.s. |
| DQI (Reasons) | –0.12 | n.s. | –0.12 | n.s. | –0.12 | n.s. |
| DQI (Personal experiences) | –0.37 | * | –0.41 | † | –0.40 | † |
| Age | 0.08 | n.s. | 0.07 | n.s. | ||
| Education | –0.04 | n.s. | –0.04 | n.s. | ||
| Gender (female = 1) | –0.04 | n.s. | –0.04 | n.s. | ||
| Area (UPZ=1) | 0.03 | n.s. | ||||
| Area (west=1) | 0.01 | n.s. | ||||
| Area (east=1) | –0.06 | n.s. | ||||
| Sample Size | 63 | 62 | 62 | |||
| AIC | 142.46 | 143.94 | 149.23 | |||
| R2 | 0.13 | 0.17 | 0.18 | |||
| Power(1–β) | 0.60 | 0.64 | 0.59 | |||
**p < 0.01, *p < 0.05, †p < 0.10.
The results of the multiple regression analysis indicated that the DQI score for ‘Personal experiences,’ which was the proportion of statements with ‘Personal experiences’ in the total number of statements, impacted the change in individual attitudes across three models for SS-Ⅰ and SS-Ⅲ, although the direction of the effect was opposite. For SS-Ⅰ (F1. conventional decision-makers), ‘Personal experiences’ positively affected the magnitude of alternation in individual attitudes, whereas ‘Opinions’ and ‘Reasons’ did not exhibit a statistically significant effect, except Model 3 in which ‘Reasons’ had a negative effect. To the contrary, for SS-Ⅲ, ‘Personal experiences’ showed a negative impact on opinion changes in all three models; participants in groups with a higher proportion of ‘Personal experiences’ were less likely to change their views on ‘national interests’ (F3). Meanwhile, no statistically significant factors were identified for SS-Ⅱ (F2. narrow localism).
Additionally, the results indicated that gender had a statistically significant difference in opinion changes; female participants, compared to males, tended not to change their attitudes on SS-I. No significant differences were observed in the number of statements by individuals, groups, or the other sociodemographic variables.
Discussion
First, it is important to note that the coefficient of determination in the regression analysis was relatively low, especially for SS-Ⅱ and SS-Ⅲ, indicating that our model does not fully explain the change in attitudes. Himmelroos and Christensen (2014) examined the relationship between discourse quality and opinion change. They reported a low coefficient of determination, below 0.2, while identifying reason-giving and issue awareness as significant predictors of opinion changes. Although the quality of deliberation and opinion changes are related, their relationship is not straightforward and requires further investigation. In addition, the statistical power was low especially for SS-Ⅱ and SS-Ⅲ in all three models; there is a high probability that a significant difference was not detected.
While acknowledging this limitation, our analysis revealed that the proportion of ‘Personal experiences’ influences attitude changes and that this effect and its direction may depend on the topic. Even though the relationship between opinion changes and quality of discourse is a central aspect of deliberative democracy, and considerable attention has been given to alternative forms of communication, the effect of storytelling on attitude changes has not been previously examined. Our research provided empirical evidence that addresses this gap and presents further complexities for research on the nexus between deliberative quality and opinion changes.
Our analysis highlighted that the impact of sharing personal experiences varied based on the topic discussed: a higher proportion of ‘Personal experiences’ in each group’s statements positively affected opinion changes for SS-I but had a negative effect for SS-Ⅲ, while it remained statistically insignificant for SS-Ⅱ. According to Polletta and Lee (2006), the frequency of storytelling is contingent on context, often less prevalent for policies or issues considered technical. In our study, while the distinctions between F1 (conventional decision-makers), F2 (narrow localism), and F3 (national interests) do not necessarily align with their arguments, F1 is more focused on trust in specific actors potentially involved in decision-making than F2 and F3. Sharing personal experiences associated with these actors might have affected the level of trust in them.
However, a qualitative analysis of statements categorized as ‘Personal experiences’ did not indicate a consistent trend. Despite these statements including experiences related to nuclear power plants, power outages, and political involvement in referendums and citizens’ meetings, there were minimal specific contents directly related to trust in conventional decision-makers. Moreover, we could not find a convincing explanation for individuals in groups with higher proportions of ‘Personal experiences’ being less prone to attitude changes for SS-Ⅲ. Consequently, further research is warranted to explore the impact of sharing personal experiences on opinion changes, particularly considering its topic-dependent nature.
These results also underscore the need for further exploration into the mechanism by which storytelling influences deliberation processes and outcomes. It is important to reiterate that the DQI items used in this analysis were not mutually exclusive. ‘Personal experiences’ included statements coded as ‘Reasons’ as well.12 Therefore, from this study alone, it remains unclear whether ‘Personal experiences’ influenced opinion changes through reason-giving or via other mechanisms.
A potential explanation for the impact on attitude changes is that sharing personal experiences may have an internal effect on shaping that person’s attitudes. To explore this hypothesis, we conducted a multiple regression analysis using the DQI scores of individual participants as explanatory variables to examine the effect of telling personal experiences on the degree of change in individual opinions (Supplementary File 1, Appendix F).13 However, ‘Personal experiences’ did not exhibit a statistically significant effect in any of the three subscale scores.
This implies that narrating personal experiences is more likely to influence opinion changes through some group dynamics. As discussed earlier, prior studies argue that storytelling builds a sense of moral community (Ryfe, 2006), crafts an identity and manages differences (Black, 2008, 2009), helps participants identify their preferences and advance unfamiliar views (Polletta & Lee, 2006), encourages perspective-taking (Muradova, 2021) and induces Deliberative Transformative Moments (Jaramillo and Steiner, 2014; Maia et al., 2020). However, our findings suggest that storytelling could have either a positive or negative effect on opinion changes. While the mentioned mechanisms might explain how personal storytelling facilitates opinion changes, they do not clarify why it sometimes inhibits them. Therefore, there is a necessity to elucidate how storytelling influences opinion changes, whether positively or negatively, on what issues, and through what mechanisms.
Finally, we found that gender had a statistically significant effect on the SS-I. Prior studies have examined the correlation between sociodemographic factors and opinion changes during deliberations. Suiter et al. (2016) reported that female participants were more likely to change their opinions. However, our results were the opposite: female participants tended not to change their attitudes compared to male participants. On the other hand, Himmelroos (2017) found that women and less-educated individuals are less influential in the deliberative process. Further research is necessary to clarify how social attributes such as gender affect opinion changes and how they relate to issues of fairness and inclusion in deliberations.
Conclusion
This study examined the impact of discourse quality on attitude changes within deliberative mini-publics, with a specific focus on deliberative polls. Our analysis revealed that the relative number of personal experiences shared in a group can significantly influence opinion changes. While previous studies have focused on the exchange of opinions based on reasons in examining the relationship between the quality of deliberation and opinion changes, our research suggests that personal storytelling could influence preference change. Nevertheless, the results yielded inconsistency; the observed effect was not consistent and appeared to vary depending on specific aspects, such as attitudes toward conventional decision-makers (SS-I) and national interests (SS-Ⅲ). Furthermore, the direction of this effect was contradictory; a higher proportion of ‘Personal experiences’ in each group’s statements positively affected opinion changes for SS-I but had a negative effect for SS-Ⅲ. This finding highlights the importance of considering the roles and functions of personal experience in deliberative processes and its intricate relationship to deliberative outcomes.
Importantly, while our findings indicate a correlation between storytelling and opinion change, they do not necessarily imply that storytelling is desirable for deliberation. As discussed earlier, while the importance of storytelling in deliberative processes has been acknowledged, its potential drawbacks have also been acknowledged (Black, 2009; Dryzek, 2000). Furthermore, the present study showed that telling personal experiences can have either a positive or negative effect on attitude changes. Therefore, it is important to recognize that our findings on the influence of personal experience on attitudes change do not make a normative claim that storytelling should be used in deliberation.
In connection with the above points, this study did not fully explore the mechanism by which personal storytelling influences attitude changes. While we have presented possible explanations, further research must explore both quantitatively and qualitatively how the various forms of storytelling affect discussions and opinion changes in deliberative forums.
Notwithstanding the importance of understanding the relationship between discourse quality and deliberation outcomes in deliberative theory, empirical evidence in this area is currently insufficient. This study aimed to address this gap by providing insights into the impact of personal storytelling on opinion changes. Despite the inconsistency of the results and the relatively poor fit of our models, this study can support the further development of deliberation theory and practice by contributing to empirical findings.
Data Accessibility Statement
The data that support the findings of this study are openly available in ‘figshare’ at https://doi.org/10.6084/m9.figshare.22229794.
Additional File
The additional file for this article can be found as follows:
Appendix A to Appendix F. DOI: https://doi.org/10.16997/jdd.1426.S1
Notes
- For a detailed description of the procedure, see Nakazawa and Tatsumi (2022). ⮭
- The main data for this study were obtained from T2 and T3 and a record of the discussion, while Nakazawa and Tatsumi (2022) used only the data of T1. ⮭
- See Supplementary File 1 (Appendix A) for the list of the 30 items. ⮭
- See Supplementary File 1 (Appendix B) for a comprehensive overview of the coding criteria used for evaluating these three items. ⮭
- Using the absolute number of statements corresponding to each item as a score would result in identical scores irrespective of the total number of statements in a group, provided that the same number of statements corresponding to each item were made. We consider this method inappropriate as an indicator of the group’s deliberation quality. ⮭
- Omaezaki city is the host municipality of the Hamaoka nuclear power plant. ⮭
- For more details, see Supplementary File 1 (Appendix C). ⮭
- Each category includes graduates, current students, and dropouts. ⮭
- See Supplementary File 1 (Appendix D) for the change in each subscale score before and after the discussion for each group. ⮭
- See Appendix E for a comprehensive presentation of the detailed results. ⮭
- Calculated as α = 0.05. ⮭
- Additionally, based on this coding data, it is not feasible to ascertain if the specific portion of a statement categorized as ‘Personal experiences’ is simultaneously coded as ‘Reasons.’ ⮭
- Note that in our primary analysis, the DQI scores were calculated as the ratio of each DQI item in the overall number of statements within each group; the score for ‘Personal experiences’ was considered a group variable. Nevertheless, to investigate the potential internal impact of sharing personal experiences on shaping an individual’s attitudes, we recalibrated the score as an individual variable: the proportion of statements coded as ‘Personal experiences’ in the total number of statements of each individual. ⮭
Ethics and Consent
This study was conducted with the approval of 17–45 of the Ethics Review Committee for Research Involving Human Subjects at Shizuoka University.
Funding Information
This work was supported by JST CREST Grant Numbers JPMJCR15E1, JPMJCR20D1, and JSPS KAKENHI Grant Numbers 18K01994, 22H01072, 23K22343.
Competing Interests
The authors have no competing interests to declare.
Author Contributions
Takashi Nakazawa led the overall study and was responsible for coding the statements. Tomoyuki Tatsumi was in charge of the data analysis. Yume Souma and Susumu Ohnuma were mainly responsible for developing the evaluation index.
References
Adams, B. E. (2014). Reason-giving in deliberative forums. Journal of Deliberative Democracy, 10(2), Article 6. DOI: http://doi.org/10.16997/jdd.208
Bächtiger, A., & Parkinson, J. (2019). Mapping and Measuring Deliberation. Oxford University Press. DOI: http://doi.org/10.1093/oso/9780199672196.001.0001
Bächtiger, A., Shikano, S., Pedrini, S., & Ryser, M. (2009). Measuring deliberation 2.0: Standards, Discourse Types, and Sequenzialization. ECPR General Conference. Available from http://ash.harvard.edu/extension/ash/docs/baechtiger.pdf.
Black, L. W. (2008). Deliberation, storytelling, and dialogic moments. Communication Theory, 18(1), 93–116. DOI: http://doi.org/10.1111/j.1468-2885.2007.00315.x
Black, L. W. (2009). Listening to the city: Difference, identity, and storytelling in online deliberative groups. Journal of Public Deliberation, 5(1), Article 4. DOI: http://doi.org/10.16997/jdd.76
Black, L. W. (2013). How people communicate during deliberative events. In T. Nabatchi, J. Gastil, M. Leighninger, & G. M. Weiksner (Eds), Democracy in Motion: Evaluating the Practice and Impact of Deliberative Civic Engagement (pp. 59–81). DOI: http://doi.org/10.1093/acprof:oso/9780199899265.003.0004
Caluwaerts, D., & Reuchamps, M. (2014). Does inter-group deliberation foster inter-group appreciation? Evidence from two experiments in Belgium. Politics, 34(2), 101–115. DOI: http://doi.org/10.1111/1467-9256.12043
Chambers, S. (2003). Deliberative democratic theory. Annual Review of Political Science, 6, 307–326. DOI: http://doi.org/10.1146/annurev.polisci.6.121901.085538
Deliberative Democracy Lab. (n.d.). Center on democracy, development and the rule of law at Stanford University. Available from https://cdd.stanford.edu/
Dryzek, J. S. (2000). Deliberative Democracy and Beyond. Oxford University Press. DOI: http://doi.org/10.1093/019925043x.001.0001
Dutwin, D. (2001). Can people talk politics? A study of deliberative democracy. (Doctoral dissertation, University of Pennsylvania, Philadelphia).
Feller, T. (2018). Personal stories told during jury Deliberations. Storytelling, Self, Society, 13(1), 94–119. DOI: http://doi.org/10.13110/storselfsoci.13.1.0094
Fishkin, J. S. (2009). When the People Speak: Deliberative Democracy and Public Consultation. Oxford University Press. DOI: http://doi.org/10.1093/acprof:osobl/9780199604432.001.0001
Gerber, M., Bächtiger, A., Fiket, I., Steenbergen, M., & Steiner, J. (2014). Deliberative and non-deliberative persuasion: Mechanisms of opinion formation in EuroPolis. European Union Politics, 15(3), 410–429. DOI: http://doi.org/10.1177/1465116514528757
Gerber, M., Bächtiger, A., Shikano, S., Reber, S., & Rohr, S. (2018). Deliberative abilities and influence in a transnational deliberative poll (EuroPolis). British Journal of Political Science, 48(4), 1093–1118. DOI: http://doi.org/10.1017/S0007123416000144
Goodin, R. E., & Niemeyer, S. J. (2003). When does deliberation begin? Internal reflection versus public discussion in deliberative democracy. Political Studies, 51(4), 627–649. DOI: http://doi.org/10.1111/j.0032-3217.2003.00450.x
Grönlund, K., Bächtiger, A., & Setälä, M. (Eds.) (2014). Deliberative Mini-publics: Involving Citizens in the Democratic Process. ECPR Press.
He, B., Breen, M. G., & Fishkin, J. S. (2021). Deliberative Democracy in Asia. Routledge. DOI: http://doi.org/10.4324/9781003102441
Himmelroos, S. (2017). Discourse quality in deliberative citizen forums – A comparison of four deliberative mini-publics. Journal of Public Deliberation, 13(1), Article 3. DOI: http://doi.org/10.16997/jdd.269
Himmelroos, S., & Christensen, H. S. (2014). Deliberation and opinion change: Evidence from a deliberative mini-public in Finland. Scandinavian Political Studies, 37(1), 41–60. DOI: http://doi.org/10.1111/1467-9477.12013
Jacquet, V., & van der Does, R. (2021). The consequences of deliberative minipublics: Systematic overview, conceptual gaps, and new directions. Representation, 57(1), 131–141. DOI: http://doi.org/10.1080/00344893.2020.1778513
Jaramillo, M. C., & Steiner, J. (2014). Deliberative transformative moments: A new concept as amendment to the discourse quality index. Journal of Public Deliberation, 10(2), Article 8. DOI: http://doi.org/10.16997/jdd.210
Karpowitz, C. F., & Raphael, C. (2014). Deliberation, Democracy, and Civic Forums: Improving Equality and Publicity. Cambridge University Press. DOI: http://doi.org/10.1017/CBO9781107110212
Knobloch, K., & Gastil, J. (2022). How deliberative experiences shape subjective outcomes: A study of fifteen minipublics from 2010–2018. Journal of Deliberative Democracy, 18(1), 1–16. DOI: http://doi.org/10.16997/jdd.942
Lindell, M. (2015). Differences in deliberative behavior between individuals with polarized and moderated opinions. A study of equality, respect and rational justification. Paper presented at the workshop The Challenges of the Exponential Diffusion of Democratic Innovations: How to Accumulate Knowledge from Case Studies? 1–21.
Luskin, R. C., Fishkin, J. S., & Jowell, R. (2002). Considered opinions: Deliberative polling in Britain. British Journal of Political Science, 32(3), 455–487. DOI: http://doi.org/10.1017/S0007123402000194
Maia, R. C. M., Cal, D., Bargas, J., & Crepalde, N. J. B. (2020). Which types of reason-giving and storytelling are good for deliberation? Assessing the discussion dynamics in legislative and citizen forums. European Political Science Review, 12(2), 113–132. DOI: http://doi.org/10.1017/S1755773919000328
Maia, R. C. M., & Hauber, G. (2020). The emotional dimensions of reason-giving in deliberative forums. Policy Sciences, 53(1), 33–59. DOI: http://doi.org/10.1007/s11077-019-09363-1
McCoy, M. L., & Scully, P. L. (2002). Deliberative dialogue to expand civic engagement: What kind of talk does democracy need? National Civic Review, 91(2), 117–135. DOI: http://doi.org/10.1002/ncr.91202
Muhlberger, P. (2006). Lessons from the Virtual Agora project: The effects of agency, identity, information, and deliberation on political knowledge. Journal of Public Deliberation, 2(1), Article 13. DOI: http://doi.org/10.16997/jdd.37
Muradova, L. (2021). Seeing the other side? Perspective-taking and reflective political judgements in interpersonal deliberation. Political Studies, 69(3), 644–664. DOI: http://doi.org/10.1177/0032321720916605
Muradova, L., Walker, H., & Colli, F. (2020). Climate change communication and public engagement in interpersonal deliberative settings: Evidence from the Irish citizens’ assembly. Climate Policy, 20(10), 1322–1335. DOI: http://doi.org/10.1080/14693062.2020.1777928
Mutz, D. C. (2008). Is deliberative democracy a falsifiable theory? Annual Review of Political Science, 11 (February), 521–538. DOI: http://doi.org/10.1146/annurev.polisci.11.081306.070308
Nakazawa, T., & Tatsumi, T. (2022). Disagreeing with ‘local agreement’: A survey of public attitudes toward restarting the Hamaoka nuclear power plant. Environmental Planning and Management, 65(1), 150–167. DOI: http://doi.org/10.1080/09640568.2021.1880378
Neblo, M. (2007). Change for the better? Linking the mechanisms of deliberative opinion change to normative theory. Paper presented at the Annual Meeting of the International Societyfor Political Psychology. Available from http://polisci.osu.edu/faculty/mneblo/papers.htm
O’Malley, E., Farrell, D. M., & Suiter, J. (2020). Does talking matter? A quasi-experiment assessing the impact of deliberation and information on opinion change. International Political Science Review, 41(3), 321–334. DOI: http://doi.org/10.1177/0192512118824459
Pinto, J., Paneth, N., Kazam, E., Kairam, R., Wallenstein, S., Rose, W., Rosenfeld, D., Schonfeld, S., Stein, I., & Witomski, T. (1988). Interobserver variability in neonatal cranial ultrasonography. Paediatric and Perinatal Epidemiology, 2(1), 43–58. DOI: http://doi.org/10.1111/j.1365-3016.1988.tb00179.x
Polletta, F., & Gardner, B. (2018). The forms of deliberative communication. In A. Bächtiger, J. S. Dryzek, J. Mansbridge., & M. E. Warren (Eds.), The Oxford Handbook of Deliberative Democracy (pp. 70–85). DOI: http://doi.org/10.1093/oxfordhb/9780198747369.013.45
Polletta, F., & Lee, J. (2006). Is telling stories good for democracy? Rhetoric in public deliberation after 9/II. American Sociological Review, 71(5), 699–723. DOI: http://doi.org/10.1177/000312240607100501
Ryfe, D. M. (2006). Narrative and deliberation in small group forums. Journal of Applied Communication Research, 34(1), 72–93. DOI: http://doi.org/10.1080/00909880500420226
Sanders, D. (2012). The effects of deliberative polling in an EU-wide experiment: Five mechanisms in search of an explanation. British Journal of Political Science, 42(3), 617–640. DOI: http://doi.org/10.1017/S0007123411000494
Sanders, L. M. (1997). Against deliberation. Political Theory, 25(3), 347–376. DOI: http://doi.org/10.1177/0090591797025003002
Smith, G. (2009). Democratic Innovations: Designing Institutions for Citizen Participation. Cambridge University Press. DOI: http://doi.org/10.1017/CBO9780511609848
Smith, G., & Setälä, M. (2018). Mini-publics and deliberative democracy. In A. Bächtiger, J. S. Dryzek, J. Mansbridge, & M. E. Warren (Eds), The Oxford Handbook of Deliberative Democracy (pp. 70–85). Oxford University Press. DOI: http://doi.org/10.1093/oxfordhb/9780198747369.013.27
Souma, Y., Yokoyama, M., Nakazawa, T., Tatsumi, T., & Ohnuma, S. (2022). The group discussion experiment on the treatment of removed low concentration soil outside Fukushima Prefecture: Contemplation of common goods and the development of the index visualizing the discourse qualities. Japanese Journal of Risk Analysis, 32(1), 11–23. DOI: http://doi.org/10.11447/jjra.SRA-0387
Steenbergen, M. R., Bächtiger, A., Spörndli, M., & Steiner, J. (2003). Measuring political deliberation: A discourse quality index. Comparative European Politics, 1(1), 21–48. DOI: http://doi.org/10.1057/palgrave.cep.6110002
Steiner, J., Bächtiger, A., Spörndli, M., & Steenbergen, M.R. (2004). Deliberative Politics in Action. Cambridge University Press. DOI: http://doi.org/10.1017/CBO9780511491153
Stromer-galley, J. (2007). Measuring deliberation’s content: A coding scheme. Journal of Public Deliberation, 3(1), Article 12. DOI: http://doi.org/10.16997/jdd.50
Suiter, J., Farrell, D. M., & O’Malley, E. (2016). When do deliberative citizens change their opinions? Evidence from the Irish citizens’ Assembly. International Political Science Review, 37(2), 198–212. DOI: http://doi.org/10.1177/0192512114544068
Sunstein, C. R. (2000). Deliberative trouble? Why groups go to extremes. Yale Law Journal, 110(1), 71–119. DOI: http://doi.org/10.2307/797587
Tanaka, A. (Ed.) (2018). Jukugi no Koyo, Jukuryo no koka [Benefits of Deliberation and Effects of Reflection]. Keiso Shobo.
Tatsumi, T., & Nakazawa, T. (2021). Public attitudes towards local agreement over the restart of nuclear power plants: A questionnaire survey on the case of Hamaoka. Transactions of the Atomic Energy Society of Japan, 20(2), 50–61. DOI: http://doi.org/10.3327/taesj.J20.006
Westwood, S. J. (2015). The role of persuasion in deliberative opinion change. Political Communication, 32(4), 509–528. DOI: http://doi.org/10.1080/10584609.2015.1017628
Wongpakaran, N., Wongpakaran, T., Wedding, D., & Gwet, K. L. (2013). A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: A study conducted with personality disorder samples. BMC Medical Research Methodology, 13(1), 1–7. DOI: http://doi.org/10.1186/1471-2288-13-61
Young, I. M. (2000). Inclusion and Democracy. Oxford University Press.