1. Introduction
Abortion discourse is often characterised as a hyper-polarised, moral-political issue infused with excessive negative campaigning, invective, and radical tactics. In the last decades, some academics came to demoralising conclusions that the abortion issue illustrates the failure of democracy to create a public sphere to effectively deliberate deeply contested and divisive issues (Ferree et al., 2002; Hunter, 1994).
Given the rise of abortion politics around the world including the United States (US) and Ireland (Field, 2018; Ziegler, 2023), this paper reports on a timely, topical study assessing the democratic qualities of contemporary abortion discourse using two deliberative criteria: civility and tolerance – two important qualities of mutual respect to build constructive disagreements (Gutmann & Thompson, 1996; Habermas, 1996, 2003). Mutual respect is especially required in abortion discourse as abortion politics involves conflicting positions with incommensurable worldviews and morality that cannot be solved with ‘rational’ persuasion (Ferree et al., 2002; Hunter, 1994). Our findings can also provide insights about the public’s capabilities to deliberate a range of other emotional and polarised culture war issues including LGBT+ rights (Hunter, 1994). The US-Ireland comparative research design enables us to identify the cross-national comparability of our findings.
This paper focuses on a single social media platform, Twitter, which contributes to facilitate digital ‘issue publics’ (Habermas, 2006), bridging between official political discourse and civil societies, and between elite media and public communications (Bruns & Highfield, 2015). Elected politicians and journalists use Twitter, more than other platforms (Elayan et al., 2020). The public use Twitter to engage in political conversations and form deliberative publics (Wright et al., 2017). This political affordance of Twitter makes the platform suitable to study the deliberative qualities of public discussions on abortion. We use two Twitter datasets mentioning the American (2020) constitutional discussions on Roe v. Wade and the Irish (2018) referendum for a constitutional amendment on abortion bans.
We develop and test a set of expectations to explain the dynamics of incivility and intolerance at micro and meso levels. On the micro-level, we analyse the demographics of those who are more likely to express incivility and intolerance in online abortion discourse. We also assess the user distribution of uncivil and intolerant tweets to investigate whether incivility and intolerance are common behaviours for all actors involved in the discourse, or unusual behaviours mobilised by a small set of particular users. On the meso-level, we examine the role of contextual factors such as the type of discussion environments in which a tweet is embedded.
Our results show that the level of incivility and intolerance expressed during the 2020 American and 2018 Irish abortion discourse on Twitter is low over time, but these uncivil and intolerant tweets generate wider public engagements than civil and tolerant tweets. Our micro-level analysis shows that pro-life1 Twitter users are associated with intolerance in both countries. We also observe the coordinated dynamics of incivility and intolerance, where a small number of hyperactive users dominate the entire uncivil and intolerant communications. In our meso-level analysis, incivility is associated with a heterogeneous discussion environment whereas intolerance is associated with homogeneous echo chambers.
Below, we explain how we operationalise civility and tolerance as two important normative criteria of deliberative democracy. We conceptualise Twitter as issue publics (Habermas, 2006). We then summarise the history of abortion discourse in the US and Ireland before we measure and analyse incivility and intolerance in the American and Irish abortion discourse on Twitter and collate the results for cross-national insights.
2. Deliberative democracy and mutual respect: Civility and tolerance
Deliberative democracy is the notion that a democratic society’s rules and conditions are justified through public debate among equal citizens. (Cohen 1989). Habermas (1996) idealises the power of rational and reasonable deliberation to solve socially shared problems. Gutmann and Thompson (1996) similarly describe deliberative democracy as the obligation to provide reasonable justification for public claims (that transcend their narrow self-interest) to deal with social conflicts.
Decisions on public affairs are made at the political centre (e.g., by governments etc) but discussions influencing the decision must incorporate peripheral voices – including autonomous citizens representing non-bureaucratic, personal and everyday experiences (Ferree et al., 2002). Habermas assumes that autonomous citizens deliberate more freely than formal-bureaucratic actors since they are free from the burden of making decisions and institutional regulations and therefore can respect the better arguments and take the viewpoints of other actors (Ferree et al., 2002). Autonomous citizens and organisations are crucial in deliberative democracy since their voices bring new social issues and perspectives to the political and public spheres, contributing to possible solutions (Habermas, 1996, 2006).
The normative standard of mutual respect is central to the deliberative model of democracy. It safeguards deliberative politics whereby public decisions are made by the strength of the arguments, and not by coercion, or prejudice. Habermas’ ‘ideal speech situation’ (1990) encapsulates normative deliberation by endorsing mutual respect despite deep disagreements. Gutmann and Thompson (1996) argue that ‘citizens do not end serious moral conflict, but […] they accept significant parts of the substantive morality of their fellow citizens to whom they may find themselves deeply opposed in other respects’ (p.89).
Based on the previous literature on deliberation, we argue that mutual respect consists of two aspects: civility and tolerance. Civility here refers to the social norms of interpersonal relationships and politeness. Incivility is a violation of the civility norm such as threatening opponents, name-calling, rudeness etc. This relates to the expectations of deliberation as civilised and reasonable discussions. This does not mean civility is tantamount to emotional detachment per se, but it nonetheless contains efforts to keep one’s negative emotions toward opponents in check.
Tolerance refers to the moral-political respect for each other with which deliberation participants actively promote and recognise significant parts of the morality of their opponents as fellow citizens (Gutmann & Thompson, 1996). Intolerance in deliberative politics means the lack of such moral-political respect for fellow citizens such as demonising the other side as something that cannot be tolerated and absolutising one’s unique ethical view as a generally binding rule for the whole society (Habermas, 2003).
Numerous insightful studies investigated diverse qualities of public deliberation such as rationality, sourcing, reasoned opinion expressions through content analysis (Stromer-Galley, 2007) as well as (in)civility (Rossini & Maia, 2021) and participant (in)equality (Maia et al., 2017). Our research aims to contribute to deliberation scholarship by presenting a novel way to assess the level of mutual respect in online issue publics through two criteria: (in)civility and (in)tolerance.
3. Twitter and issue publics
Deliberative democracy emphasises the importance of autonomous groups of citizens in the public spheres. Habermas views public spheres as a space where public actors join public debates, influencing the state’s decision-making. Habermas (2006) acknowledges the continued structural transformation of the public spheres due to the Internet. In his words (2006), the Internet reintroduces deliberative elements and reactivate an egalitarian public of writers and readers who were relegated as audience in mass media communications.
His 2006 essay also discusses the Internet and fragmentation of issue publics: ‘the rise of millions of fragmented chat rooms […] lead[s] to the fragmentation of large but politically focused mass audiences into a huge number of isolated issue publics’ (p.422). Other scholars further explored ‘issue publics.’ Dahlgren (2009) argues that issue publics ‘emerge, exist for varying durations, and then eventually dissolve’ as the public debates move on (p.74). Bruns and Highfield (2015) argue that issue publics form around short-lived topics and events that exist in connection with wider public spheres around specific themes. While the wider public sphere addresses the long-term issue of abortion, issue publics would form around specific policy initiatives and other short-term aspects that ignite public debates (Bruns & Highfield, 2015).
Social media sites such as Twitter can facilitate the formation of issue publics (Bruns & Highfield, 2015). For many on social media, Twitter is a preferred platform for news, information consumption and political participation such as campaigning, activism, and deliberation (Wright et al., 2017). Hashtags on Twitter enable users to join discussions and form issue publics (Bruns & Highfield, 2015). Abortion discussions on American and Irish Twitterspheres constitute abortion issue publics that invite clusters of citizens and public actors (e.g., politicians). However, Twitter research is not without its limitations and biases and therefore any findings in this paper should not be automatically generalised to infer attitudes and behaviours of the American and Irish population as a whole.
Furthermore, deliberation consists of rational and normative standards that are difficult to be rigorously met in online discussions. Nonetheless, assessing the volume and dynamics of incivility and intolerance in online abortion discourse is worthy of academic attention to understand how the public engages in everyday opinion expressions and discussions about highly emotional and polarised moral-political issues like abortion.
4. Abortion discourse in the US and Ireland
This paper evaluates the degree to which the 2020 American and 2018 Irish abortion discourse on Twitter satisfies or violates the deliberative criteria of civility and tolerance. The US and Ireland are chosen as case-studies because their constitutions on abortion were among mainstream political priorities in recent years.
In the US in 1973, Roe v. Wade was a landmark decision in which the Supreme Court ruled that the Constitution protects pregnant people’s liberty to terminate pregnancy without excessive state restrictions (Ferree et al., 2002). Since Roe, American pro-life movements have tried to introduce creative ways to legislate abortion restrictions or even to reverse Roe. With the revival of Christian nationalism, reactionary backlash, and the presidential election of Donald Trump, American pro-life movements gained new momentum (Norris & Inglehart, 2019; Ziegler, 2023). In 2020, there were heated public discussions about the precarious future of Roe alongside debates around the implications and future of abortion bans and Planned Parenthood, the American reproductive healthcare provider. The newly appointed and nominated conservative Supreme Court Justices during the Trump administration such as Justice Kavanaugh and Justice Barrett ignited public discussions about overturning Roe. At the time of writing this paper, the new conservative majority Supreme Court overturned Roe in June 2022 (Ziegler, 2023).
In Ireland, a historically Catholic country, abortion was banned for a long time. In 1983, the 8th amendment of the Irish Constitution was implemented to ‘acknowledge the right to life of the unborn.’ There were several more referendums to decide further restrictions on abortion such as to remove the threat of suicide as a ground for legal abortion (12th and 25th amendments) and restrict pregnant people’s rights to travel for abortion and to access information about abortion clinics abroad (13th and 14th amendments) (Field, 2018).
Abortion remained absent from Irish mainstream priorities in past decades (Field, 2018). The death of Savita Halappanavar in October 2012, who died of sepsis after being denied a legal abortion, ignited the debate which led to another abortion referendum in 2018 (Field, 2018).
Formal deliberation is another characteristic of the Irish abortion discourse. In 2016, the Citizens’ Assembly was established to deliberate on how to change the Irish abortion constitution. After weeks of deliberation, the Citizens’ Assembly provided recommendations to the Oireachtas (Irish legislature) (Field, 2018). The use of Citizens’ Assemblies in Ireland (in both the marriage equality referendum in 2014 and the abortion referendum in 2016) highlight that Ireland is at the forefront of deliberative experiments among European democracies and that Irish citizens have hands-on experiences with the learn-deliberate model (Escobar & Elstub, 2017; Suiter, 2018).
In May 2018, Ireland held a referendum for the 36th amendment of the constitution to repeal the 8th, 13th, and 14th amendments of its Constitution. Repeal won by a margin of 66.4% to 33.6% (Field, 2018). That year, the President of Ireland signed the Health Regulation of Termination of Pregnancy Act, defining the circumstances and processes within which abortion is legally performed in Ireland.
It must be noted that the American abortion discourse in 2020 and Irish discourse in 2018 are not directly comparable, given that the former case is more of a spectator discourse where the public expressed and reacted to news stories about policy and legal decisions whereas the latter is a voter communication centred around the referendum with involvement of institutionalised actors like politicians, activists, etc. Also, the Irish abortion discourse involved formal deliberation whereas the American discourse largely consisted of unorganised public opinions. Nonetheless, interrogating the differences between American and Irish abortion discourse on Twitter can yield insights about the cross-national comparability of our findings.
Our first research question investigates the volume of incivility and intolerance in the American and Irish abortion discourse on Twitter.
RQ1: How much incivility and intolerance exist in the American and Irish abortion discourse on Twitter?
Given the more radical and polarised nature of abortion politics in the US involving violence and radicalism (Ferree et al., 2002; Hunter, 1994) in comparison with Ireland with public experiences with formal deliberation on abortion constitutions (Suiter, 2018), we expect that American data will involve more anti-deliberative behaviours than the Irish data.
H1: American tweets about abortion contain more incivility and intolerance than Irish tweets.
5. Individual and contextual dynamics of incivility and intolerance
In addition to comparing the national differences, we investigate the micro and meso factors influencing the expressions of incivility and intolerance in abortion discourse. We aim to ascertain whether a specific stance on abortion significantly correlates with anti-deliberative behaviours. While uncivil, disruptive communication tactics are used by both liberal and conservative political actors (Berry & Sobieraj, 2013), social and political intolerance are strongly associated with right-wing politics (Norris & Inglehart, 2019). Given the history of anti-abortion violence and its ties to reactionary politics (Ferree et al., 2002; Norris & Inglehart, 2019), we hypothesise a stronger association between pro-life stance and intolerance. Our hypothesis 2 anticipates that:
H2: In both American and Irish abortion discourse on Twitter, pro-life tweets are more likely to be intolerant than pro-choice tweets whereas the association between abortion issue positions and incivility are not as strongly correlated.
Moreover, we explore the dynamics and distribution of these uncivil and intolerant behaviours; whether they are common across all participants or specifically mobilised and coordinated by a smaller set of hyperactive users. Whether anti-deliberative behaviours are common will have different implications for our analysis of deliberative qualities of abortion discourse on Twitter. Theocharis et al. (2020) analyse the coordination dynamics of Twitter incivility directed at American politicians and conclude that incivility is a common behaviour among angry, frustrated citizens to criticise and insult politicians. Since our Twitter dataset is specifically confined to abortion issue publics, we examine whether the degree of mobilisation of incivility and intolerance is concentrated on few but committed-hyperactive actors.
H3: In both American and Irish Twitter, incivility and intolerance are common public behaviours, equally expressed by all participants.
Our hypotheses and research questions lead to another question. Regardless of whether a specific type of user expresses incivility and intolerance often, are there specific interactive contexts that trigger larger outbursts of incivility and intolerance? The interactive communicative contexts of Twitter discussions, in combination with the individual characteristics of Twitter users, may drive more incivility and intolerance than in other communicative contexts.
Several studies investigated how discussion environments influence the level of incivility and intolerance in user communications. Rains et al. (2017) find that the proportion of in- and out-group members in online news comment sections affects conservative and liberal users’ tendencies to be uncivil to their opponents. Rossini (2020) finds that incivility in Facebook news comments is associated with expressions of disagreements whereas intolerance occurs in homogeneous discussions about minorities.
In this paper, we focus on whether incivility and intolerance occur in inter-group engagements or in self-imposed echo chambers2 where users only communicate with already like-minded users (Bruns, 2019; Garrett, 2017). Our expectation is that incivility occurs more when users are actively engaging and arguing with one another in a heterogeneous discussion environment whereas intolerance occurs more frequently within homogeneous echo chambers, when users are sharing their absolutist and extreme views with like-minded people (Rossini, 2020).
H4: In both American and Irish abortion discourse on Twitter, incivility is more likely to be expressed in a heterogeneous environment and intolerance in a homogeneous echo chamber.
6. Data and methods
6.1. Data collection
Our American dataset contains 6,305,107 tweets collected between 4 March and 20 October 2020 from the Twitter stream API. The five search keywords for the data collection are Roe v. Wade, abortion ban, Planned Parenthood, pro-choice, and pro-life. For the Irish dataset, we obtained a previously shared dataset from the Harvard Dataverse (Littman, 2018). This dataset contains 2,279,396 tweets collected between 13 April and 4 June 2018 from the Twitter filter stream API, collected via 63 hashtags: 32 pro-choice (e.g., #togetherforyes); 14 pro-life (e.g., #savethe8th, #lovebothvoteno); and 7 hashtags that are neutral or ambiguous in relation to abortion (e.g., #8thref, #hometovote). The rehydrated Irish dataset contains 1,842,370 tweets: the lost tweets are due to how Twitter’s APIs and platform work.3 It should be noted that our American and Irish datasets were collected through different methods due to the differences in their nature. Since the Irish case was a referendum voter discourse, popular hashtags mobilised by institutional campaign actors were easily findable. In contrast, since the American Roe discourse was a spectator discourse, it lacked established hashtags and we instead had to rely on popular keywords to collect our data.
We removed spam by excluding tweets with nine or more hashtags, given that spam tweets tend to include more hashtags than legitimate tweets (Chen et al., 2015). We also removed tweets containing spam words (e.g., PS4, giveaway) that we identified through manual qualitative coding of small random samples (Jashinsky et al., 2014). We also identified tweets where Irish abortion referendum hashtags were used to discuss abortion-irrelevant issues such as #voteyes to discuss a Scottish Independence referendum. We also removed non-English tweets. Post-filtering, the final size of the American dataset is 6,054,670 tweets and the Irish dataset is 1,707,979 tweets.
The limitation of a single platform study should be acknowledged. The Twitter userbase is generally skewed towards men, younger, educated, and urban-residing adults, who tend to have higher interests in politics than the general public (Barbera & Rivero, 2015). Hence, these results should not be translated as a general, representative summary of American and Irish abortion discourse at a national level, nor can they predict user behaviours across other social platforms.
6.2. Automatically classifying tweets
To address our research questions and hypotheses, we develop a computer-assisted classifier to automatically categorise our big Twitter datasets. In this section, we summarise how we computationally label incivility, intolerance, abortion issue positions, gender, and discussion contexts. We also explain how we measure Twitter coordination dynamics to see the distribution of uncivil and intolerant tweets.
6.2.1. Classifying incivility and intolerance
We applied a lexicon-based classification to detect incivility and intolerance in our American and Irish Twitter datasets. The classifier includes word stems to successfully capture the variations of one uncivil or intolerant word and phrases (e.g., fuck.* to capture fucking, fucker, etc.).
Muddiman et al. (2019)’s work to build ‘manually validated organic dictionaries’ of incivility was a reference for our classifier-building process. Manual labelling and validation enable the successful construction of incivility and intolerance classifiers that are theoretically derived and context-dependent (Muddiman et al., 2019). There are incivility studies that employ more state-of-the-art techniques such as supervised machine learning (ML) (Theocharis et al., 2020), but we have chosen a lexicon-based approach over ML mainly for economic efficiency of time and efforts (Mukhtar et al., 2018). Furthermore, this semi-supervised, lexicon-based approach eliminates the chances of algorithmic biases known in many ML algorithms by providing a level of transparency (Bender et al., 2021). This is not to quibble with other ML-based works but to illustrate a rationale for choosing a manually validated lexicon-based approach (Muddiman et al., 2019).
For each tweet, the coder labelled incivility and intolerance using a coding scheme4 inspired by Ferree et al. (2002), Coe et al. (2014) and Rossini (2020). Table 1 provides examples of incivility and intolerance. The two variables have binary categories.
Examples of incivility and intolerance.
| Form of incivility (interpersonal disrespect/rudeness) | Form of intolerance (moral disrespect) |
| Name-calling/insults: ‘you’re a bigoted cunt,’ ‘his moronic views’ Vulgarity: ‘fucking bullshit,’ ‘what the fuck,’ ‘crap’ Pejorative remark about how one communicates: ‘Quit crying over the spilt milk of …,’ ‘what a stupid argument’ |
Equation of abortion to intolerable extreme ideologies including genocide, the Nazi holocaust, slavery Demonisation: ‘They are devils,’ ‘evil,’ ‘satanic’ Dehumanisation: ‘These pest/rats/roaches would be exterminated’ Absolutism of one viewpoint as a generally and reciprocally binding morality: e.g., women/foetus have a right to freedom/life, it is immoral to infringe on it. Calling for violence, threats: e.g., saying that people should be punched/killed, buildings should be vandalised/arsoned |
To build incivility and intolerance classifiers, the authors manually coded 6,000 tweets from the American dataset and 5,700 tweets from the Irish dataset to create a list of uncivil and intolerant word stems and expressions. We then checked the performance (i.e., accuracy, precision, recall) of the classifiers and added more expressions to enhance the performance. The full manual labelling of these training samples was conducted by one coder, but multiple coders actively partook in the process of coding scheme development and inter-coder reliability tests. The coding scheme for manual qualitative labelling achieves substantial agreements in terms of both Cohen’s kappa and percentage agreement. The abortion issue position has 91.3% agreement and 0.83 kappa. The incivility variable has 93% agreement and 0.76 kappa, and intolerance has 94% agreement and 0.74 kappa. Percentage agreement was used to complement the overly conservative measure of beyond-chance agreement of Cohen’s kappa (Joyce, 2013).
The final version of the American incivility classifier achieved the mean accuracy of 85.8% with the mean precision of 91% and mean recall of 73.9%. The American intolerance classifier yielded the mean accuracy of 86.5% with the mean precision of 82% and the mean recall of 67.8%. The mean accuracy of the Irish incivility classifier was 93.7% with the mean precision of 86% and mean recall of 61.8%. The mean performance of the Irish intolerance classifier was 98.3% accuracy, 99.2% precision, and 70.8% recall. Recall performances were lower than accuracy or precision as it was difficult to identify every creative and nuanced form of incivility and intolerance in big data. For instance, studies have discussed the difficulty of automatic sarcasm detection on Twitter: sarcasm alone can cause a 50% drop in accuracy when automatically detecting sentiment (Sykora et al., 2020). Minimising false-positives provides a reliable standard for assessing the performance of natural language classifiers (Muddiman et al., 2019). The performance of our lexicon-based classifier matches the performance of ML classifiers such as in Theocharis et al. (2020).
6.2.2. Classifying abortion issue position
We coded the abortion issue positions of Twitter users into pro-choice, pro-life, and unidentifiable (either ambiguous or neutral to abortion). Since our American and Irish datasets were collected through different methods, two different tactics were used.
For the American dataset, we classified abortion issue positions using the tf-idf5 scores of the top 5,000 predictive n-grams6 for pro-choice and pro-life tweets. The training set for the American abortion position classification contained 3,000 tweets and the testing set contained 708 tweets. We then extracted n-grams with the top 2,500 tf-idf values from each pro-choice and pro-life position. Many n-grams contained the same tf-idf values, and therefore the top n-grams with the top 2,500 tf-idf values were more than 2,500: 2,803 pro-choice n-grams and 5,178 pro-life n-grams. Most of these top n-grams were bigrams and trigrams.
We classified American tweets based on the pro-life and pro-choice n-grams. When a tweet included pro-life/pro-choice n-grams only, the tweet was classified as a pro-life/pro-choice tweet. If a tweet contained both pro-life and pro-choice n-grams, the tweet was classified as position neutral or ambiguous. If a tweet did not contain any pro-life or pro-choice n-grams, it was also classified as position neutral or ambiguous. When evaluated on the testing set, the accuracy was 71.8%;higher than the baseline accuracy and as high as Sharma et al. (2017)’s ML ideology classifier.
Since the Irish dataset (Littman, 2018) was collected through 53 key hashtags with clear ideological indications, we classified the Irish dataset into pro-life, pro-choice, and unidentifiable positions following the ideological hashtags. If a tweet contained only pro-choice (or pro-life) hashtags, it was labelled as pro-choice (or pro-life). If a tweet did not have either pro-choice or pro-life hashtags, it was coded as unidentifiable. If a tweet had both pro-choice and pro-life hashtags simultaneously, it was also coded as unidentifiable. We randomly sampled 1,000 tweets from the Irish dataset and compared the hashtag-based automatic classification and manual classification. The accuracy of the hashtag-based classifier is 84%. This accuracy is as high as the state-of-the-art ML classification of abortion ideologies in Sharma et al. (2017) therefore our simple and intuitive classification models can perform as accurately as the best accuracy of a more complex ML classifier.
6.2.3. Classifying gender and anonymity
We used an R package for Automatic Gender Recognition (AGR): Gender (Mullen, 2021). This package predicts gender by name from historical data, typically population data gathered by American Census and Social Security data (Mullen, 2021). Gender automatically allocated Twitter users in our datasets into three nominal categories: women, men, N/A (gender unidentifiable; e.g., anonyms like ‘Old Glory’).
We treat gender as a control variable to our regression model since it can have confounding relationships with users’ attitudes and behaviours to incivility and intolerance in online abortion discussions. First, gender can moderate the relationship between abortion issue positions, incivility, and intolerance. Studies suggest that reactionary voices against feminism and abortion rights often come from conservative men (Norris & Inglehart, 2019). Wang and Silva (2018) find that in online abortion discussions, gender interacts with the broader opinion climate which then affects the relationship between exposure to incivility and increased negative emotions. Women feel more negative emotions when they see incivility directed at their side and when their view is in a disadvantageous position whereas men’s emotion or willingness to partake online are not affected by opinion climate (Wang & Silva, 2018). Anonymity is associated with disinhibition and hence increased tendency to be uncivil without worrying about consequences (Coe et al., 2014).
Since our specific operationalisation of gender and anonymity carries limitations and hence impedes a statistical investigation with predictive power, we choose to have gender and anonymity as control variables. First and foremost, it is impossible to verify the ‘real’ gender of the account holder with the plausible exception of blue-ticked, verified accounts. Second, the operationalisation of anonymity based on the first name has limitations. Someone could name their account with obvious pseudonyms such as John Doe, but would be classified as men and non-anonymous based on our chosen method. Nonetheless, predicting gender based on first names has been used in studies involving big online datasets (King & Frederickson, 2021). By using this method, we are not assuming that all individuals are correctly gendered, but this provides insight into gender’s effects in aggregate.
6.2.4. Classifying discussion contexts
Our Twitter datasets collected through Twitter APIs contain meta-information about communication contexts, indicating whether a post was a tweet, retweet, quote, or reply. We treat tweet networks built through retweets and quotes as a proxy to a homogeneous discussion environment and treat replies as a proxy to a heterogeneous discussion environment. Our operationalisation is built on previous Twitter studies that have found that Twitter networks built through quotes and retweets contain agreements and homophily whereas the networks built through replies have more disagreements (Garimella et al., 2016). We do not assert that this simplified method correctly classifies every tweet, but it provides insights about the effect of discussion environments in aggregate.
6.2.5. Measuring Twitter coordination dynamics
We measured coordinated dynamics of incivility and intolerance by computing the Gini index for inequality in the distribution of tweets by users (Theocharis et al., 2020). Gini coefficients can capture coordination dynamics on Twitter: whether a small set of users are leading uncivil and intolerant communications. A higher Gini coefficient indicates greater levels of inequality, and this enables us to analyse whether a discussion is dominated by a small number of participants or whether it is open to a wide range of participants with similar levels of involvement. A Gini coefficient close to 1 means that most uncivil/intolerant tweets are posted by a single user, while a coefficient close to 0 would imply that all users are equally likely to post such content.
After classifying the tweets, logistic regression was employed to examine the relationship between independent variables (abortion issue positions, discussion contexts) and dependent variables (incivility and intolerance). Gender and anonymity are added to our model as control variables. We must also note the limitation in our regression analysis, like many others using Twitter data (e.g., Theocharis et al., 2020), where we treat each tweet as statistically independent. Although we do include a simplified approximation of the intricate relationships in a discussion, where the variable ‘discussion context’ (i.e., original/initial tweet in a thread, reply, etc.) provides an approximate proxy for the relationship between an uncivil/intolerant tweet and the discussion context it is embedded in.
7. Results
Table 2 shows how the American and Irish Twitter datasets are constructed. Both datasets contain large portions of tweets posted by gender-unidentifiable users. One supposition might be that many users might have been concerned about voicing their views in the abortion issue publicly with their gender-identifiable names. This observation could prompt future studies to examine who uses anonymity and why.
Descriptive characteristics of the American and Irish Twitter datasets.
| Category | US | Ireland | |||||
|---|---|---|---|---|---|---|---|
| Number | Per cent | Number | Per cent | ||||
| Gender | Women | 1,722,144 | 28.40% | 633,267 | 37.10% | ||
| Men | 1,449,422 | 23.90% | 415,413 | 24.30% | |||
| Unidentifiable | 2,883,104 | 47.60% | 659,299 | 38.60% | |||
| Issue position | Pro-life | 1,593,090 | 30.60% | 198,906 | 11.60% | ||
| Pro-choice | 1,852,511 | 26.30% | 1,064,159 | 62.30% | |||
| Unidentifiable | 2,609,069 | 43.10% | 444,914 | 26% | |||
| Tweet affordance | Original tweet | 384,013 | 6.30% | 189,505 | 11.10% | ||
| Retweet | 4,415,076 | 72.90% | 1,121,611 | 65.70% | |||
| Quote | 509,140 | 8.40% | 341,506 | 20% | |||
| Reply | 746,441 | 12.30% | 55,357 | 3.20% | |||
| Discussion context | Homogeneous | 5,091,400 | 81.30% | 1,463,117 | 85.70% | ||
| Heterogeneous | 746,441 | 12.30% | 55,357 | 3.20% | |||
| Incivility | 929,032 | 15.30% | 153,028 | 9% | |||
| Intolerance | 606,542 | 10% | 29,469 | 1.70% | |||
Some notable differences between the two are: (1) the Irish dataset contains a larger portion of tweets by women than the American dataset. (2) The Irish dataset is predominantly pro-choice whereas the American data has a near-equal mix of pro- and anti-abortion tweets. (3) The Irish data consists of retweets followed by quotes and original tweets whereas American data consists of retweets followed by replies and quotes. Lastly, (4) the American dataset contains more incivility and intolerance than the Irish data (H1), which could be explained by the radical and violent history of abortion activism in the US (Ferree et al., 2002; Hunter, 1994).
Overall, both countries’ Twitter abortion discourse involves a small amount of uncivil and intolerant communications. 15.3% and 10% of American tweets and 9% and 1.7% of Irish tweets contain some form of incivility and intolerance (RQ1). The occurrences of incivility and intolerance in our datasets are not particularly higher than the volume of incivility and intolerance found in other online political public spheres (Coe et al., 2014; Rossini, 2020; Theocharis et al., 2020). This indicates that the impact of incivility and intolerance is felt more strongly than their quantitative shares of the total abortion discourse (Ferree et al., 2002).
We calculated Pearson’s correlation coefficients (r) between incivility and intolerance. The coefficient between incivility and intolerance is 0.22 in the American data and 0.09 in the Irish data, showing positive linear trends. However, the sizes of both coefficients are weak.
Table 3 below shows the distribution of uncivil and intolerant tweets in the American and Irish datasets. To investigate the reach of the uncivil and intolerant tweets, we calculated the mean, median, and standard deviations (SDs) of the amount of public engagements the uncivil and/or intolerant tweets generate.
Number of tweets in the civility-tolerance matrix and the means, medians, and SDs of the number of retweets the tweets received.
| US | Civility | Incivility |
|---|---|---|
| Tolerance | 4,754,878 (mean: 16257.55, median: 559, SD: 32649.92) |
693,250 (mean: 2064.442, median: 192, SD: 4219.13) |
| Intolerance | 370,760 (mean: 5425.39, median: 910, SD: 7916.27) |
235,782 (mean: 7360.46, median: 1372, SD: 11575.41) |
| Ireland | ||
| Tolerance | 1,534,078 (mean: 701.05, median: 26, SD: 2477.87) |
144,432 (mean: 2341.51, median: 124, SD: 4854.19) |
| Intolerance | 20,873 (mean: 327.67, median: 56, SD: 492.82) |
8,596 (mean: 324.18, median: 84, SD: 427.14) |
Different types of tweets got more engagements in American and Irish Twitterspheres. In the American dataset, tweets that do not contain incivility nor intolerance earn the most retweets on average. As for the median of retweets, uncivil and intolerant tweets got most public engagements. In the Irish case, uncivil tweets earned the most retweets both in terms of their mean and median values. Considering median measures is important here, since distributions of retweets can be skewed by a small proportion of extremely popular tweets and a large portion of tweets that earned zero retweets. This indicates that although incivility and intolerance are uncommon behaviours, they still provoke huge volumes of engagements thereby influencing the atmosphere and direction of online public debates as well as the wider discussions on news media and elite discourse outside of Twitter.
To test other research hypotheses, a logistic regression analysis examines the relationship between diverse demographic characteristics and communicative contexts, incivility, and intolerance. Log odds are transformed to odds ratios to facilitate interpretation and 95% confidence intervals (CIs) are reported in addition to p-values. Reporting CIs is especially important to complement the error-prone nature of p-values in big data studies (Lin et al., 2013). Results of the regression analysis are in Table 4.
Odds ratio and 95% confidence intervals of incivility and intolerance in American and Irish abortion discourse on Twitter.
| US | Ireland | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Incivility | Intolerance | Incivility | Intolerance | ||||||
| OR | CI [2.5%, 97.5%] | OR | CI [2.5%, 97.5%] | OR | CI [2.5%, 97.5%] | OR | CI [2.5%, 97.5%] | ||
| (Intercept) | 0.11*** | [0.11, 0.11] | 0.03*** | [0.03, 0.03] | 0.09*** | [0.09, 0.09] | 0.00*** | [0.00, 0.01] | |
| Pro-life | 2.19*** | [2.18, 2.21] | 8.53*** | [8.45, 8.61] | 0.89*** | [0.87, 0.90] | 12.59*** | [12.21, 12.99] | |
| Position unidentifiable | 1.46*** | [1.46, 1.47] | 2.21*** | [2.18, 2.23] | 1.42*** | [1.41, 1.44] | 3.81*** | [3.69, 3.94] | |
| Heterogeneous environment | 1.26*** | [1.25, 1.26] | 0.55*** | [0.55, 0.56] | 1.11*** | [1.07, 1.14] | 0.85*** | [0.8, 0.91] | |
| Gender men | 1.06*** | [1.05, 1.06] | 1.09*** | [1.08, 1.1] | 1.11*** | [1.1, 1.13] | 1.4*** | [1.36, 1.45] | |
| Gender unidentifiable | 1.07*** | [1.07, 1.08] | 1.05*** | [1.04, 1.06] | 1.1*** | [1.09, 1.12] | 1.35*** | [1.32, 1.39] | |
Signif. Codes: 0 ‘***’.
The results show that the pro-life issue position is strongly related to more intolerance in both countries. The odds of intolerant communications for pro-life tweets are 8.53 times higher in the American data and 12.59 times higher in the Irish dataset than pro-choice tweets. The relationship between incivility and abortion issue positions shows conflicting results. The odds of uncivil tweets for pro-life tweets are 2.19 times higher in the US, but 0.89 times lower in Ireland. This shows that in the American Twittersphere, pro-life tweets are more likely to be uncivil and intolerant, while in the Irish Twittersphere, pro-life tweets are more associated with intolerance whereas pro-choice tweets express more incivility (H2 supported).
Furthermore, our micro-level analysis shows that uncivil and intolerant Twitter communications are mobilised by a small number of ‘superparticipants’ (Graham & Wright, 2014) (H3 rejected). In the American dataset, the Gini coefficient for the tweet distribution is 0.6. The level of coordination is larger for uncivil and intolerant tweets: 0.85 for incivility and 0.92 for intolerance. In the American abortion discourse on Twitter, the top 1% of most active users are responsible for 27.1% of the total incivility and 39.5% of total intolerance. This pattern is also observed for our Irish dataset where the Gini coefficient for the tweet distribution is 0.73. The Gini coefficient for incivility is 0.88 and the coefficient for intolerance is 0.97. The top 1% of the most active Irish users express 41% of the total incivility and almost entire intolerance in the dataset. Figure 1 visualises the Lorenz curves for the distribution of total tweets, incivility, and intolerance in the two datasets.
Lorenz curves for user coordination dynamics in American and Irish datasets.
The regression model also reveals a meso-level relationship between discussion environments and incivility and intolerance. The odds of incivility are 1.26 times higher in the American data and 1.11 times higher in the Irish dataset when a tweet is embedded in a heterogeneous environment where Twitter users are actively disagreeing with other tweets. In contrast, the odds of intolerance become 0.55 times lower in the American data and 0.85 times lower in the Irish data, meaning that intolerance is strongly associated with a homogeneous environment, lacking disagreements and inter-group engagements (H4 confirmed).
8. Concluding remarks and future directions
We explore Twitter discussions as a facilitator of abortion issue public, evaluating the public capabilities for deliberating highly divisive culture war issues like abortion (Hunter, 1994) while observing to the normative deliberative criteria of mutual respect. We operationalise mutual respect with two concepts: civility (interpersonal politeness) and tolerance (moral-political respect). Studying American and Irish cases together provides findings on the potential similarities and transferability of the results.
Despite choosing an emotive and polarised issue like abortion, the Twitter discussions in the US and Ireland are mostly civil and tolerant. This contrasts with perceived culture wars and antipathy in their abortion debates (Brownstein, 2021) suggesting that the impact of incivility and intolerance outstrips their actual representation in the discourse (Ferree et al., 2002). However, we find that uncivil and intolerant tweets generate more public engagements, potentially influencing a wider discussion.
We also find that American abortion discussions contain more incivility and significantly more intolerance than Irish discussions, potentially due to America’s history of violent abortion activism (Ferree et al., 2002) and Ireland’s experience with deliberative minipublics over contentious issues e.g., same-sex marriage (Suiter, 2018). This invites further research into whether the institutional uses of deliberative politics increase deliberative qualities of online discussions globally.
The regression analysis reveals that uncivil and intolerant communications on Twitter are associated with different demographic and discussion contexts. The micro-level analysis identified intolerance as a distinctively pro-life behaviour in both American and Irish Twitterspheres, supporting existing literature linking uncivil society with reactionary right-wing discourse (Norris & Inglehart, 2019).
In our meso-level analysis, we find that incivility prevails in heterogeneous environments whereas intolerance occurs within self-selected echo chambers (Rossini, 2020). This indicates the necessity to intervene in spiralling intolerant echo chambers.
Another noteworthy micro-level finding is the coordinated dynamics of uncivil and intolerant behaviours (Theocharis et al., 2020). Uncivil and intolerant discourse on Twitter is largely expressed and circulated by a small set of ‘superparticipants’ (Graham & Wright, 2014). The overrepresentation of hyperpartisan voices on social media leads to a distorted understanding of how polarised and extreme the opposingside is, making politically moderate citizens leave the online discourse (Bail, 2021) creating subsequent inequalities in policy outcomes where hyper-partisan voices are strongly represented and heard more by politicians and policymakers in digital spaces (Kleiner, 2018). It is crucial for online deliberation to curtail the capabilities of these superparticipants distorting and radicalising the overall online political discourse. The development of a meaningful typology of hyperactive users to generate a deeper understanding of who these people are and how they operate is needed.
Notes
- To respectfully represent both sides, we have chosen to use the collective labels that each side prefers to self-identify with (YouGov, 2022). Labels such as pro-abortion and anti-abortion are criticised by some pro-choice and pro-life groups for being reductive (Pew, 2008; Save the Storks, 2019). ⮭
- Echo chambers refer to environments where people only encounter and engage with information and opinions that are similar to their own. Our understanding focuses on self-imposed echo chambers (users actively choose to consume certain information) than algorithmically-driven understanding of the phenomenon (Bruns, 2019). ⮭
- Data rehydration is the process of importing data so it can be readily accessed and used. This is done using tools, e.g., Twarc2 (https://twarc-project.readthedocs.io/en/latest/twarc2_en_us/#hydrate). For official information, see Twitter developer at https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/overview. ⮭
- Our coding scheme is available here: https://docs.google.com/document/d/1EUgPQn3l5wsHiur-79KfiulWYv0FdrwLLt9cHnzzUbw/edit?usp=sharing. ⮭
- Tf-idf stands for Term frequency and Inverse Document Frequency matrix. Tf-idf weighs common words and rare words, calculating the importance of a word to a specific document (Silge & Robinson, 2017). ⮭
- In natural language processing, an n-gram is a parsing of a text into a number of sequences of words or characters. For sequences of words, unigrams (n = 1) that can be generated from a sentence ‘How are you’ are ‘how,’ ‘are,’ ‘you’; possible bigrams (n = 2) are ‘how are’ and ‘are you.’ ⮭
Open Source
The R code and dictionary text files are freely available via the Open Science Framework (OSF) and can be accessed by following this link: https://osf.io/u6gw2/.
Competing Interests
The authors have no competing interests to declare.
Publishers Note
Table 3 was updated on 20th October 2023 to re-insert one row that was missing from the original publication. This row specifies that the bottom two rows are regarding the Irish case.
References
1 Bail, C. (2021). Breaking the social media prism. Princeton University Press. DOI: http://doi.org/10.1515/9780691216508
2 Barbera, P., & Rivero, G. (2015). Understanding the political representativeness of twitter users. Social Science Computer Review, 33(6), 712–729. DOI: http://doi.org/10.1177/0894439314558836
3 Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 acm conference on fairness, accountability, and transparency, 610–623. DOI: http://doi.org/10.1145/3442188.3445922
4 Berry, J. M., & Sobieraj, S. (2013). The outrage industry: Political opinion media and the new incivility. Oxford University Press.
5 Brownstein, R. (2021, 25 May). A culture war with real consequences is coming. CNN. https://edition.cnn.com/2021/05/25/politics/abortion-mississippi-supreme-court-culture-war/index.html
6 Bruns, A. (2019). It’s not the technology, stupid: How the ‘Echo Chamber’ and ‘Filter Bubble’ metaphors have failed us. Paper presented at the 2019 IAMCR conference, 1–12.
7 Bruns, A., & Highfield, T. (2015). Is Habermas on Twitter? Social media and the public sphere. In the Routledge companion to social media and politics, 56–73. Routledge. DOI: http://doi.org/10.4324/9781315716299-5
8 Chen, C., Zhang, J., Chen, X., Xiang, Y., & Zhou, W. (2015). 6 million spam tweets: A large ground truth for timely Twitter spam detection. In 2015 ieee international conference on communications (icc), 7065–7070. DOI: http://doi.org/10.1109/ICC.2015.7249453
9 Coe, K., Kenski, K., & Rains, S. A. (2014). Online and uncivil? Patterns and determinants of incivility in newspaper website comments. Journal of Communication, 64(4), 658–679. DOI: http://doi.org/10.1111/jcom.12104
10 Cohen, J. (1989). “Deliberation and Democratic Legitimacy.” In Alan Hamlin and Philip Pettit (eds), The Good Polity. Cambridge, MA: Blackwell, 17–34.
11 Dahlgren, P. (2009). Media and political engagement. Cambridge: Cambridge University Press.
12 Elayan, S., Sykora, M., & Jackson, T. (2020). “His Tweets Speak for Themselves”: An Analysis of Donald Trump’s Twitter Behavior. The International Journal of Interdisciplinary Civic and Political Studies, 15(1), 11. DOI: http://doi.org/10.18848/2327-0071/CGP/v15i01/11-35
13 Escobar, O., & Elstub, S. (2017). Forms of mini-publics: An introduction to deliberative innovations in democratic practice. New Democracy. Research and Development Note, 4.
14 Ferree, M. M., Gamson, W. A., & Rucht, D. (2002). Shaping abortion discourse: Democracy and the public sphere in Germany and the United States. Cambridge University Press. DOI: http://doi.org/10.1017/CBO9780511613685
15 Field, L. (2018). The abortion referendum of 2018 and a timeline of abortion politics in Ireland to date. Irish Political Studies, 33(4), 608–628. DOI: http://doi.org/10.1080/07907184.2018.1500461
16 Garimella, K., Weber, I., & De Choudhury, M. (2016). Quote RTs on Twitter: Usage of the New Feature for Political Discourse. In Proceedings of the 8th acm conference on web science, 200–204. DOI: http://doi.org/10.1145/2908131.2908170
17 Garrett, R. K. (2017). The “echo chamber” distraction: Disinformation campaigns are the problem, not audience fragmentation. Journal of Applied Research in Memory and Cognition, 6(4), 370–376. DOI: http://doi.org/10.1016/j.jarmac.2017.09.011
18 Graham, T., & Wright, S. (2014). Discursive Equality and Everyday Talk Online: The Impact of “Superparticipants”. Journal of Computer-Mediated Communication, 19(3), 625–642. DOI: http://doi.org/10.1111/jcc4.12016
19 Gutmann, A., & Thompson, D. (1996). Democracy and disagreement. Cambridge, MA: Belknap Press.
20 Habermas, J. (1990). Moral consciousness and communicative action. MIT press.
21 Habermas, J. (1996). Between facts and norms (W. Rehg, Trans.). The MIT Press. DOI: http://doi.org/10.7551/mitpress/1564.001.0001
22 Habermas, J. (2003). Intolerance and discrimination. International Journal of Constitutional Law, 1(1), 2–12. DOI: http://doi.org/10.1093/icon/1.1.2
23 Habermas, J. (2006). Political communication in media society: Does democracy still enjoy an epistemic dimension? The impact of normative theory on empirical research. Communication Theory, 16(4), 411–426. DOI: http://doi.org/10.1111/j.1468-2885.2006.00280.x
24 Hunter, J. D. (1994). Before the shooting begins. Simon and Schuster.
25 Jashinsky, J., Burton, S. H., Hanson, C. L., West, J., Giraud-Carrier, C., Barnes, M. D., & Argyle, T. (2014). Tracking suicide risk factors through Twitter in the US. Crisis. DOI: http://doi.org/10.1027/0227-5910/a000234
26 Joyce, M. (2013). Picking the best intercoder reliability statistic for your digital activism content analysis. Digital Activism Research Project, 243(11).
27 King, M. M., & Frederickson, M. E. (2021). The pandemic penalty: the gendered effects of COVID-19 on scientific productivity. Socius, 7, 23780231211006977. DOI: http://doi.org/10.1177/23780231211006977
28 Kleiner, T. (2018). Public opinion polarisation and protest behaviour. European Journal of Political Research, 57(4), 941–962. DOI: http://doi.org/10.1111/1475-6765.12260
29 Lin, M., Lucas, H. C., Jr., & Shmueli, G. (2013). Research commentary—too big to fail: large samples and the p-value problem. Information Systems Research, 24(4), 906–917. DOI: http://doi.org/10.1287/isre.2013.0480
30 Littman, J. (2018). Ireland 8th Tweet Ids. Harvard Dataverse, V1. DOI: http://doi.org/10.7910/DVN/PYCLPE
31 Maia, R., Cal, D., Bargas, J. K., Oliveira, V. V., Rossini, P. G., & Sampaio, R. C. (2017). Authority and deliberative moments: Assessing equality and inequality in deeply divided groups. Journal of Public Deliberation, 13(2). DOI: http://doi.org/10.16997/jdd.283
32 Muddiman, A., McGregor, S. C., & Stroud, N. J. (2019). (Re) claiming our expertise: Parsing large text corpora with manually validated and organic dictionaries. Political Communication, 36(2), 214–226. DOI: http://doi.org/10.1080/10584609.2018.1517843
33 Mukhtar, N., Khan, M. A., & Chiragh, N. (2018). Lexicon-based approach outperforms Supervised Machine Learning approach for Urdu Sentiment Analysis in multiple domains. Telematics and Informatics, 35(8), 2173–2183. DOI: http://doi.org/10.1016/j.tele.2018.08.003
34 Mullen, L. (2021). gender: Predict gender from names using historical data. r package version 0.5.4.1000. https://github.com/ropensci/gender/.
35 Norris, P., & Inglehart, R. (2019). Cultural backlash: Trump, Brexit, and authoritarian populism. Cambridge University Press. DOI: http://doi.org/10.1017/9781108595841
36 Pew Research Centre. (2008). Pro-Choice Does Not Mean Pro-Abortion: An Argument for Abortion Rights Featuring the Rev. Carlton Veazey. https://www.pewresearch.org/religion/2008/09/30/pro-choice-does-not-mean-pro-abortion-an-argument-for-abortion-rights-featuring-the-rev-carlton-veazey/
37 Rains, S. A., Kenski, K., Coe, K., & Harwood, J. (2017). Incivility and political identity on the Internet: Intergroup factors as predictors of incivility in discussions of news online. Journal of Computer-Mediated Communication, 22(4), 163–178. DOI: http://doi.org/10.1111/jcc4.12191
38 Rossini, P. (2020). Beyond incivility: Understanding patterns of uncivil and intolerant discourse in online political talk. Communication Research, 0093650220921314. DOI: http://doi.org/10.1177/0093650220921314
39 Rossini, P., & Maia, R. C. M. (2021). Characterizing Disagreement in Online Political Talk: Examining Incivility and Opinion Expression on News Websites and Facebook in Brazil. Journal of Deliberative Democracy, 17(1), pp. 90–104. DOI: http://doi.org/10.16997/10.16997/jdd.967
40 Save the Storks. (2019). The difference between being pro-life and anti-abortion. https://savethestorks.com/2019/03/the-difference-between-being-pro-life-and-anti-abortion/
41 Sharma, E., Saha, K., Ernala, S. K., Ghoshal, S., & De Choudhury, M. (2017). Analyzing ideological discourse on social media: A case study of the abortion debate. In Proceedings of the 2017 international conference of the computational social science society of the Americas, 1–8. DOI: http://doi.org/10.1145/3145574.3145577
42 Silge, J., & Robinson, D. (2017). Text mining with r: A tidy approach. O’Reilly Media, Inc. DOI: http://doi.org/10.16997/jdd.50
43 Stromer-Galley, J. (2007). Measuring deliberation’s content: A coding scheme. Journal of Deliberative Democracy, 3(1).
44 Suiter, J. (2018). Deliberation in action–Ireland’s abortion referendum. Political Insight, 9(3), 30–32. DOI: http://doi.org/10.1177/2041905818796576
45 Sykora, M., Elayan, S., & Jackson, T. W. (2020). A qualitative analysis of sarcasm, irony and related# hashtags on twitter. Big Data & Society, 7(2), 2053951720972735. DOI: http://doi.org/10.1177/2053951720972735
46 Theocharis, Y., Barbera, P., Fazekas, Z., & Popa, S. A. (2020). The Dynamics of Political Incivility on Twitter. SAGE Open, 10(2), 1–15. DOI: http://doi.org/10.1177/2158244020919447
47 Wang, M. Y., & Silva, D. E. (2018). A slap or a jab: An experiment on viewing uncivil political discussions on Facebook. Computers in Human Behavior, 81, 73–83. DOI: http://doi.org/10.1016/j.chb.2017.11.041
48 Wright, S., Graham, T., & Jackson, D. (2017). Third space and everyday online political talk: Deliberation, polarisation, avoidance. In: The 67th Annual Conference of the International Communication Association, 25–29 May 2017, San Diego, USA.
49 YouGov. (2022). Pro-life, anti-abortion, or anti-choice: what label Americans use for people who oppose abortion. https://today.yougov.com/topics/politics/articles-reports/2022/06/17/pro-life-anti-abortion-or-anti-choice-what-label-a?amp;amp=
50 Ziegler, M. (2023). Roe: The History of a National Obsession. Yale University Press. DOI: http://doi.org/10.12987/9780300271874
