<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.2 20120330//EN" "http://jats.nlm.nih.gov/publishing/1.2/JATS-journalpublishing1.dtd">
<!--<?xml-stylesheet type="text/xsl" href="article.xsl"?>-->
<article article-type="research-article" dtd-version="1.2" xml:lang="en" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<front>
<journal-meta>
<journal-id journal-id-type="issn">2634-0488</journal-id>
<journal-title-group>
<journal-title>Journal of Deliberative Democracy</journal-title>
</journal-title-group>
<issn pub-type="epub">2634-0488</issn>
<publisher>
<publisher-name>University of Westminster Press</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.16997/jdd.1413</article-id>
<article-categories>
<subj-group>
<subject>Research article</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>Deliberative Qualities of Online Abortion Discourse: Incivility and Intolerance in the American and Irish Abortion Discussions on Twitter</article-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name>
<surname>Oh</surname>
<given-names>Dayei</given-names>
</name>
<email>dayei.oh@helsinki.fi</email>
<xref ref-type="aff" rid="aff-1">1</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Elayan</surname>
<given-names>Suzanne</given-names>
</name>
<xref ref-type="aff" rid="aff-2">2</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Sykora</surname>
<given-names>Martin</given-names>
</name>
<xref ref-type="aff" rid="aff-2">2</xref>
</contrib>
</contrib-group>
<aff id="aff-1"><label>1</label>Helsinki Institute for Social Sciences and Humanities, The University of Helsinki, FI</aff>
<aff id="aff-2"><label>2</label>Centre for Information Management, Loughborough University, UK</aff>
<pub-date publication-format="electronic" date-type="pub" iso-8601-date="2023-07-21">
<day>21</day>
<month>07</month>
<year>2023</year>
</pub-date>
<pub-date pub-type="collection">
<year>2023</year>
</pub-date>
<volume>19</volume>
<issue>1</issue>
<fpage>1</fpage>
<lpage>12</lpage>
<history>
<date date-type="received" iso-8601-date="2023-01-23">
<day>23</day>
<month>01</month>
<year>2023</year>
</date>
<date date-type="accepted" iso-8601-date="2023-06-20">
<day>20</day>
<month>06</month>
<year>2023</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright: &#x00A9; 2023 The Author(s)</copyright-statement>
<copyright-year>2023</copyright-year>
<license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/4.0/">
<license-p>This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See <uri xlink:href="http://creativecommons.org/licenses/by/4.0/">http://creativecommons.org/licenses/by/4.0/</uri>.</license-p>
</license>
</permissions>
<self-uri xlink:href="https://delibdemjournal.org/articles/10.16997/jdd.1413/"/>
<abstract>
<p>This paper provides a big-data-scale assessment of the deliberative qualities of online abortion discussions on Twitter in the United States (2020) and Ireland (2018) by specifically focusing on two standards: civility and tolerance for constructive disagreements. Using diverse computational methods and classification, our regression analysis provides mixed evaluations. We find that incivility and intolerance are uncommon behaviours in American and Irish abortion discourse on Twitter, but we also find that these anti-deliberative behaviours are (a) generating more engagements and thereby distorting the overall discussion atmosphere; (b) largely coming from the pro-life tweets; (c) dominated by a small set of hyperactive participants; and that (d) intolerant users tend to communicate within homogeneous echo chambers. Our results indicate that it is crucial for online deliberation to curtail the capabilities of these superparticipants distorting and radicalising the overall online political discourse. By studying two national contexts, our results provide comparability of our findings and insights that can improve our understanding of other contentious and polarised issues more broadly.</p>
</abstract>
<kwd-group>
<kwd>abortion discourse</kwd>
<kwd>online deliberation</kwd>
<kwd>incivility</kwd>
<kwd>intolerance</kwd>
<kwd>computational social science</kwd>
<kwd>Twitter</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<sec>
<title>1. Introduction</title>
<p>Abortion discourse is often characterised as a hyper-polarised, moral-political issue infused with excessive negative campaigning, invective, and radical tactics. In the last decades, some academics came to demoralising conclusions that the abortion issue illustrates the failure of democracy to create a public sphere to effectively deliberate deeply contested and divisive issues (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>; <xref ref-type="bibr" rid="B24">Hunter, 1994</xref>).</p>
<p>Given the rise of abortion politics around the world including the United States (US) and Ireland (<xref ref-type="bibr" rid="B15">Field, 2018</xref>; <xref ref-type="bibr" rid="B50">Ziegler, 2023</xref>), this paper reports on a timely, topical study assessing the democratic qualities of contemporary abortion discourse using two deliberative criteria: civility and tolerance &#8211; two important qualities of mutual respect to build constructive disagreements (<xref ref-type="bibr" rid="B19">Gutmann &amp; Thompson, 1996</xref>; <xref ref-type="bibr" rid="B21">Habermas, 1996</xref>, <xref ref-type="bibr" rid="B22">2003</xref>). Mutual respect is especially required in abortion discourse as abortion politics involves conflicting positions with incommensurable worldviews and morality that cannot be solved with &#8216;rational&#8217; persuasion (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>; <xref ref-type="bibr" rid="B24">Hunter, 1994</xref>). Our findings can also provide insights about the public&#8217;s capabilities to deliberate a range of other emotional and polarised culture war issues including LGBT+ rights (<xref ref-type="bibr" rid="B24">Hunter, 1994</xref>). The US-Ireland comparative research design enables us to identify the cross-national comparability of our findings.</p>
<p>This paper focuses on a single social media platform, Twitter, which contributes to facilitate digital &#8216;issue publics&#8217; (<xref ref-type="bibr" rid="B23">Habermas, 2006</xref>), bridging between official political discourse and civil societies, and between elite media and public communications (<xref ref-type="bibr" rid="B7">Bruns &amp; Highfield, 2015</xref>). Elected politicians and journalists use Twitter, more than other platforms (<xref ref-type="bibr" rid="B12">Elayan et al., 2020</xref>). The public use Twitter to engage in political conversations and form deliberative publics (<xref ref-type="bibr" rid="B48">Wright et al., 2017</xref>). This political affordance of Twitter makes the platform suitable to study the deliberative qualities of public discussions on abortion. We use two Twitter datasets mentioning the American (<xref ref-type="bibr" rid="B12">2020</xref>) constitutional discussions on <italic>Roe v. Wade</italic> and the Irish (<xref ref-type="bibr" rid="B15">2018</xref>) referendum for a constitutional amendment on abortion bans.</p>
<p>We develop and test a set of expectations to explain the dynamics of incivility and intolerance at micro and meso levels. On the micro-level, we analyse the demographics of those who are more likely to express incivility and intolerance in online abortion discourse. We also assess the user distribution of uncivil and intolerant tweets to investigate whether incivility and intolerance are common behaviours for all actors involved in the discourse, or unusual behaviours mobilised by a small set of particular users. On the meso-level, we examine the role of contextual factors such as the type of discussion environments in which a tweet is embedded.</p>
<p>Our results show that the level of incivility and intolerance expressed during the 2020 American and 2018 Irish abortion discourse on Twitter is low over time, but these uncivil and intolerant tweets generate wider public engagements than civil and tolerant tweets. Our micro-level analysis shows that pro-life<xref ref-type="fn" rid="n1">1</xref> Twitter users are associated with intolerance in both countries. We also observe the coordinated dynamics of incivility and intolerance, where a small number of hyperactive users dominate the entire uncivil and intolerant communications. In our meso-level analysis, incivility is associated with a heterogeneous discussion environment whereas intolerance is associated with homogeneous echo chambers.</p>
<p>Below, we explain how we operationalise civility and tolerance as two important normative criteria of deliberative democracy. We conceptualise Twitter as issue publics (<xref ref-type="bibr" rid="B23">Habermas, 2006</xref>). We then summarise the history of abortion discourse in the US and Ireland before we measure and analyse incivility and intolerance in the American and Irish abortion discourse on Twitter and collate the results for cross-national insights.</p>
</sec>
<sec>
<title>2. Deliberative democracy and mutual respect: Civility and tolerance</title>
<p>Deliberative democracy is the notion that a democratic society&#8217;s rules and conditions are justified through public debate among equal citizens. (<xref ref-type="bibr" rid="B10">Cohen 1989</xref>). Habermas (<xref ref-type="bibr" rid="B21">1996</xref>) idealises the power of rational and reasonable deliberation to solve socially shared problems. Gutmann and Thompson (<xref ref-type="bibr" rid="B19">1996</xref>) similarly describe deliberative democracy as the obligation to provide reasonable justification for public claims (that transcend their narrow self-interest) to deal with social conflicts.</p>
<p>Decisions on public affairs are made at the political centre (e.g., by governments etc) but discussions influencing the decision must incorporate peripheral voices &#8211; including autonomous citizens representing non-bureaucratic, personal and everyday experiences (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>). Habermas assumes that autonomous citizens deliberate more freely than formal-bureaucratic actors since they are free from the burden of making decisions and institutional regulations and therefore can respect the better arguments and take the viewpoints of other actors (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>). Autonomous citizens and organisations are crucial in deliberative democracy since their voices bring new social issues and perspectives to the political and public spheres, contributing to possible solutions (<xref ref-type="bibr" rid="B21">Habermas, 1996</xref>, <xref ref-type="bibr" rid="B23">2006</xref>).</p>
<p>The normative standard of mutual respect is central to the deliberative model of democracy. It safeguards deliberative politics whereby public decisions are made by the strength of the arguments, and not by coercion, or prejudice. Habermas&#8217; &#8216;ideal speech situation&#8217; (<xref ref-type="bibr" rid="B20">1990</xref>) encapsulates normative deliberation by endorsing mutual respect despite deep disagreements. Gutmann and Thompson (<xref ref-type="bibr" rid="B19">1996</xref>) argue that &#8216;citizens do not end serious moral conflict, but [&#8230;] they accept significant parts of the substantive morality of their fellow citizens to whom they may find themselves deeply opposed in other respects&#8217; (p.89).</p>
<p>Based on the previous literature on deliberation, we argue that mutual respect consists of two aspects: civility and tolerance. Civility here refers to the social norms of interpersonal relationships and politeness. Incivility is a violation of the civility norm such as threatening opponents, name-calling, rudeness etc. This relates to the expectations of deliberation as civilised and reasonable discussions. This does not mean civility is tantamount to emotional detachment per se, but it nonetheless contains efforts to keep one&#8217;s negative emotions toward opponents in check.</p>
<p>Tolerance refers to the moral-political respect for each other with which deliberation participants actively promote and recognise significant parts of the morality of their opponents as fellow citizens (<xref ref-type="bibr" rid="B19">Gutmann &amp; Thompson, 1996</xref>). Intolerance in deliberative politics means the lack of such moral-political respect for fellow citizens such as demonising the other side as something that cannot be tolerated and absolutising one&#8217;s unique ethical view as a generally binding rule for the whole society (<xref ref-type="bibr" rid="B22">Habermas, 2003</xref>).</p>
<p>Numerous insightful studies investigated diverse qualities of public deliberation such as rationality, sourcing, reasoned opinion expressions through content analysis (<xref ref-type="bibr" rid="B43">Stromer-Galley, 2007</xref>) as well as (in)civility (<xref ref-type="bibr" rid="B39">Rossini &amp; Maia, 2021</xref>) and participant (in)equality (<xref ref-type="bibr" rid="B31">Maia et al., 2017</xref>). Our research aims to contribute to deliberation scholarship by presenting a novel way to assess the level of mutual respect in online issue publics through two criteria: (in)civility and (in)tolerance.</p>
</sec>
<sec>
<title>3. Twitter and issue publics</title>
<p>Deliberative democracy emphasises the importance of autonomous groups of citizens in the public spheres. Habermas views public spheres as a space where public actors join public debates, influencing the state&#8217;s decision-making. Habermas (<xref ref-type="bibr" rid="B23">2006</xref>) acknowledges the continued structural transformation of the public spheres due to the Internet. In his words (<xref ref-type="bibr" rid="B23">2006</xref>), the Internet reintroduces deliberative elements and reactivate an egalitarian public of writers and readers who were relegated as audience in mass media communications.</p>
<p>His 2006 essay also discusses the Internet and fragmentation of issue publics: &#8216;the rise of millions of fragmented chat rooms [&#8230;] lead[s] to the fragmentation of large but politically focused mass audiences into a huge number of isolated issue publics&#8217; (p.422). Other scholars further explored &#8216;issue publics.&#8217; Dahlgren (<xref ref-type="bibr" rid="B11">2009</xref>) argues that issue publics &#8216;emerge, exist for varying durations, and then eventually dissolve&#8217; as the public debates move on (p.74). Bruns and Highfield (<xref ref-type="bibr" rid="B7">2015</xref>) argue that issue publics form around short-lived topics and events that exist in connection with wider public spheres around specific themes. While the wider public sphere addresses the long-term issue of abortion, issue publics would form around specific policy initiatives and other short-term aspects that ignite public debates (<xref ref-type="bibr" rid="B7">Bruns &amp; Highfield, 2015</xref>).</p>
<p>Social media sites such as Twitter can facilitate the formation of issue publics (<xref ref-type="bibr" rid="B7">Bruns &amp; Highfield, 2015</xref>). For many on social media, Twitter is a preferred platform for news, information consumption and political participation such as campaigning, activism, and deliberation (<xref ref-type="bibr" rid="B48">Wright et al., 2017</xref>). Hashtags on Twitter enable users to join discussions and form issue publics (<xref ref-type="bibr" rid="B7">Bruns &amp; Highfield, 2015</xref>). Abortion discussions on American and Irish Twitterspheres constitute abortion issue publics that invite clusters of citizens and public actors (e.g., politicians). However, Twitter research is not without its limitations and biases and therefore any findings in this paper should not be automatically generalised to infer attitudes and behaviours of the American and Irish population as a whole.</p>
<p>Furthermore, deliberation consists of rational and normative standards that are difficult to be rigorously met in online discussions. Nonetheless, assessing the volume and dynamics of incivility and intolerance in online abortion discourse is worthy of academic attention to understand how the public engages in everyday opinion expressions and discussions about highly emotional and polarised moral-political issues like abortion.</p>
</sec>
<sec>
<title>4. Abortion discourse in the US and Ireland</title>
<p>This paper evaluates the degree to which the 2020 American and 2018 Irish abortion discourse on Twitter satisfies or violates the deliberative criteria of civility and tolerance. The US and Ireland are chosen as case-studies because their constitutions on abortion were among mainstream political priorities in recent years.</p>
<p>In the US in 1973, <italic>Roe v. Wade</italic> was a landmark decision in which the Supreme Court ruled that the Constitution protects pregnant people&#8217;s liberty to terminate pregnancy without excessive state restrictions (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>). Since <italic>Roe</italic>, American pro-life movements have tried to introduce creative ways to legislate abortion restrictions or even to reverse <italic>Roe</italic>. With the revival of Christian nationalism, reactionary backlash, and the presidential election of Donald Trump, American pro-life movements gained new momentum (<xref ref-type="bibr" rid="B35">Norris &amp; Inglehart, 2019</xref>; <xref ref-type="bibr" rid="B50">Ziegler, 2023</xref>). In 2020, there were heated public discussions about the precarious future of <italic>Roe</italic> alongside debates around the implications and future of abortion bans and Planned Parenthood, the American reproductive healthcare provider. The newly appointed and nominated conservative Supreme Court Justices during the Trump administration such as Justice Kavanaugh and Justice Barrett ignited public discussions about overturning <italic>Roe</italic>. At the time of writing this paper, the new conservative majority Supreme Court overturned <italic>Roe</italic> in June 2022 (<xref ref-type="bibr" rid="B50">Ziegler, 2023</xref>).</p>
<p>In Ireland, a historically Catholic country, abortion was banned for a long time. In 1983, the 8th amendment of the Irish Constitution was implemented to &#8216;acknowledge the right to life of the unborn.&#8217; There were several more referendums to decide further restrictions on abortion such as to remove the threat of suicide as a ground for legal abortion (12th and 25th amendments) and restrict pregnant people&#8217;s rights to travel for abortion and to access information about abortion clinics abroad (13th and 14th amendments) (<xref ref-type="bibr" rid="B15">Field, 2018</xref>).</p>
<p>Abortion remained absent from Irish mainstream priorities in past decades (<xref ref-type="bibr" rid="B15">Field, 2018</xref>). The death of Savita Halappanavar in October 2012, who died of sepsis after being denied a legal abortion, ignited the debate which led to another abortion referendum in 2018 (<xref ref-type="bibr" rid="B15">Field, 2018</xref>).</p>
<p>Formal deliberation is another characteristic of the Irish abortion discourse. In 2016, the Citizens&#8217; Assembly was established to deliberate on how to change the Irish abortion constitution. After weeks of deliberation, the Citizens&#8217; Assembly provided recommendations to the Oireachtas (Irish legislature) (<xref ref-type="bibr" rid="B15">Field, 2018</xref>). The use of Citizens&#8217; Assemblies in Ireland (in both the marriage equality referendum in 2014 and the abortion referendum in 2016) highlight that Ireland is at the forefront of deliberative experiments among European democracies and that Irish citizens have hands-on experiences with the learn-deliberate model (<xref ref-type="bibr" rid="B13">Escobar &amp; Elstub, 2017</xref>; <xref ref-type="bibr" rid="B44">Suiter, 2018</xref>).</p>
<p>In May 2018, Ireland held a referendum for the 36th amendment of the constitution to repeal the 8th, 13th, and 14th amendments of its Constitution. Repeal won by a margin of 66.4% to 33.6% (<xref ref-type="bibr" rid="B15">Field, 2018</xref>). That year, the President of Ireland signed the Health Regulation of Termination of Pregnancy Act, defining the circumstances and processes within which abortion is legally performed in Ireland.</p>
<p>It must be noted that the American abortion discourse in 2020 and Irish discourse in 2018 are not directly comparable, given that the former case is more of a spectator discourse where the public expressed and reacted to news stories about policy and legal decisions whereas the latter is a voter communication centred around the referendum with involvement of institutionalised actors like politicians, activists, etc. Also, the Irish abortion discourse involved formal deliberation whereas the American discourse largely consisted of unorganised public opinions. Nonetheless, interrogating the differences between American and Irish abortion discourse on Twitter can yield insights about the cross-national comparability of our findings.</p>
<p>Our first research question investigates the volume of incivility and intolerance in the American and Irish abortion discourse on Twitter.</p>
<disp-quote>
<p>RQ1: How much incivility and intolerance exist in the American and Irish abortion discourse on Twitter?</p>
</disp-quote>
<p>Given the more radical and polarised nature of abortion politics in the US involving violence and radicalism (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>; <xref ref-type="bibr" rid="B24">Hunter, 1994</xref>) in comparison with Ireland with public experiences with formal deliberation on abortion constitutions (<xref ref-type="bibr" rid="B44">Suiter, 2018</xref>), we expect that American data will involve more anti-deliberative behaviours than the Irish data.</p>
<disp-quote>
<p>H1: American tweets about abortion contain more incivility and intolerance than Irish tweets.</p>
</disp-quote>
</sec>
<sec>
<title>5. Individual and contextual dynamics of incivility and intolerance</title>
<p>In addition to comparing the national differences, we investigate the micro and meso factors influencing the expressions of incivility and intolerance in abortion discourse. We aim to ascertain whether a specific stance on abortion significantly correlates with anti-deliberative behaviours. While uncivil, disruptive communication tactics are used by both liberal and conservative political actors (<xref ref-type="bibr" rid="B4">Berry &amp; Sobieraj, 2013</xref>), social and political intolerance are strongly associated with right-wing politics (<xref ref-type="bibr" rid="B35">Norris &amp; Inglehart, 2019</xref>). Given the history of anti-abortion violence and its ties to reactionary politics (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>; <xref ref-type="bibr" rid="B35">Norris &amp; Inglehart, 2019</xref>), we hypothesise a stronger association between pro-life stance and intolerance. Our hypothesis 2 anticipates that:</p>
<disp-quote>
<p>H2: In both American and Irish abortion discourse on Twitter, pro-life tweets are more likely to be intolerant than pro-choice tweets whereas the association between abortion issue positions and incivility are not as strongly correlated.</p>
</disp-quote>
<p>Moreover, we explore the dynamics and distribution of these uncivil and intolerant behaviours; whether they are common across all participants or specifically mobilised and coordinated by a smaller set of hyperactive users. Whether anti-deliberative behaviours are common will have different implications for our analysis of deliberative qualities of abortion discourse on Twitter. Theocharis et al. (<xref ref-type="bibr" rid="B46">2020</xref>) analyse the coordination dynamics of Twitter incivility directed at American politicians and conclude that incivility is a common behaviour among angry, frustrated citizens to criticise and insult politicians. Since our Twitter dataset is specifically confined to abortion issue publics, we examine whether the degree of mobilisation of incivility and intolerance is concentrated on few but committed-hyperactive actors.</p>
<disp-quote>
<p>H3: In both American and Irish Twitter, incivility and intolerance are common public behaviours, equally expressed by all participants.</p>
</disp-quote>
<p>Our hypotheses and research questions lead to another question. Regardless of whether a specific type of user expresses incivility and intolerance often, are there specific interactive contexts that trigger larger outbursts of incivility and intolerance? The interactive communicative contexts of Twitter discussions, in combination with the individual characteristics of Twitter users, may drive more incivility and intolerance than in other communicative contexts.</p>
<p>Several studies investigated how discussion environments influence the level of incivility and intolerance in user communications. Rains et al. (<xref ref-type="bibr" rid="B37">2017</xref>) find that the proportion of in- and out-group members in online news comment sections affects conservative and liberal users&#8217; tendencies to be uncivil to their opponents. Rossini (<xref ref-type="bibr" rid="B38">2020</xref>) finds that incivility in Facebook news comments is associated with expressions of disagreements whereas intolerance occurs in homogeneous discussions about minorities.</p>
<p>In this paper, we focus on whether incivility and intolerance occur in inter-group engagements or in self-imposed echo chambers<xref ref-type="fn" rid="n2">2</xref> where users only communicate with already like-minded users (<xref ref-type="bibr" rid="B6">Bruns, 2019</xref>; <xref ref-type="bibr" rid="B17">Garrett, 2017</xref>). Our expectation is that incivility occurs more when users are actively engaging and arguing with one another in a heterogeneous discussion environment whereas intolerance occurs more frequently within homogeneous echo chambers, when users are sharing their absolutist and extreme views with like-minded people (<xref ref-type="bibr" rid="B38">Rossini, 2020</xref>).</p>
<disp-quote>
<p>H4: In both American and Irish abortion discourse on Twitter, incivility is more likely to be expressed in a heterogeneous environment and intolerance in a homogeneous echo chamber.</p>
</disp-quote>
</sec>
<sec>
<title>6. Data and methods</title>
<sec>
<title>6.1. Data collection</title>
<p>Our American dataset contains 6,305,107 tweets collected between 4 March and 20 October 2020 from the Twitter stream API. The five search keywords for the data collection are <italic>Roe v. Wade</italic>, abortion ban, Planned Parenthood, pro-choice, and pro-life. For the Irish dataset, we obtained a previously shared dataset from the Harvard Dataverse (<xref ref-type="bibr" rid="B30">Littman, 2018</xref>). This dataset contains 2,279,396 tweets collected between 13 April and 4 June 2018 from the Twitter filter stream API, collected via 63 hashtags: 32 pro-choice (e.g., #togetherforyes); 14 pro-life (e.g., #savethe8th, #lovebothvoteno); and 7 hashtags that are neutral or ambiguous in relation to abortion (e.g., #8thref, #hometovote). The rehydrated Irish dataset contains 1,842,370 tweets: the lost tweets are due to how Twitter&#8217;s APIs and platform work.<xref ref-type="fn" rid="n3">3</xref> It should be noted that our American and Irish datasets were collected through different methods due to the differences in their nature. Since the Irish case was a referendum voter discourse, popular hashtags mobilised by institutional campaign actors were easily findable. In contrast, since the American Roe discourse was a spectator discourse, it lacked established hashtags and we instead had to rely on popular keywords to collect our data.</p>
<p>We removed spam by excluding tweets with nine or more hashtags, given that spam tweets tend to include more hashtags than legitimate tweets (<xref ref-type="bibr" rid="B8">Chen et al., 2015</xref>). We also removed tweets containing spam words (e.g., PS4, giveaway) that we identified through manual qualitative coding of small random samples (<xref ref-type="bibr" rid="B25">Jashinsky et al., 2014</xref>). We also identified tweets where Irish abortion referendum hashtags were used to discuss abortion-irrelevant issues such as #voteyes to discuss a Scottish Independence referendum. We also removed non-English tweets. Post-filtering, the final size of the American dataset is 6,054,670 tweets and the Irish dataset is 1,707,979 tweets.</p>
<p>The limitation of a single platform study should be acknowledged. The Twitter userbase is generally skewed towards men, younger, educated, and urban-residing adults, who tend to have higher interests in politics than the general public (<xref ref-type="bibr" rid="B2">Barbera &amp; Rivero, 2015</xref>). Hence, these results should not be translated as a general, representative summary of American and Irish abortion discourse at a national level, nor can they predict user behaviours across other social platforms.</p>
</sec>
<sec>
<title>6.2. Automatically classifying tweets</title>
<p>To address our research questions and hypotheses, we develop a computer-assisted classifier to automatically categorise our big Twitter datasets. In this section, we summarise how we computationally label incivility, intolerance, abortion issue positions, gender, and discussion contexts. We also explain how we measure Twitter coordination dynamics to see the distribution of uncivil and intolerant tweets.</p>
<sec>
<title>6.2.1. Classifying incivility and intolerance</title>
<p>We applied a lexicon-based classification to detect incivility and intolerance in our American and Irish Twitter datasets. The classifier includes word stems to successfully capture the variations of one uncivil or intolerant word and phrases (e.g., fuck.* to capture fuck<italic>ing</italic>, fuck<italic>er</italic>, etc.).</p>
<p>Muddiman et al. (<xref ref-type="bibr" rid="B32">2019</xref>)&#8217;s work to build &#8216;manually validated organic dictionaries&#8217; of incivility was a reference for our classifier-building process. Manual labelling and validation enable the successful construction of incivility and intolerance classifiers that are theoretically derived and context-dependent (<xref ref-type="bibr" rid="B32">Muddiman et al., 2019</xref>). There are incivility studies that employ more state-of-the-art techniques such as supervised machine learning (ML) (<xref ref-type="bibr" rid="B46">Theocharis et al., 2020</xref>), but we have chosen a lexicon-based approach over ML mainly for economic efficiency of time and efforts (<xref ref-type="bibr" rid="B33">Mukhtar et al., 2018</xref>). Furthermore, this semi-supervised, lexicon-based approach eliminates the chances of algorithmic biases known in many ML algorithms by providing a level of transparency (<xref ref-type="bibr" rid="B3">Bender et al., 2021</xref>). This is not to quibble with other ML-based works but to illustrate a rationale for choosing a manually validated lexicon-based approach (<xref ref-type="bibr" rid="B32">Muddiman et al., 2019</xref>).</p>
<p>For each tweet, the coder labelled incivility and intolerance using a coding scheme<xref ref-type="fn" rid="n4">4</xref> inspired by Ferree et al. (<xref ref-type="bibr" rid="B14">2002</xref>), Coe et al. (<xref ref-type="bibr" rid="B9">2014</xref>) and Rossini (<xref ref-type="bibr" rid="B38">2020</xref>). <xref ref-type="table" rid="T1">Table 1</xref> provides examples of incivility and intolerance. The two variables have binary categories.</p>
<table-wrap id="T1">
<label>Table 1</label>
<caption>
<p>Examples of incivility and intolerance.</p>
</caption>
<table>
<tr>
<td colspan="2"><hr/></td>
</tr>
<tr>
<td align="left" valign="top">Form of incivility (interpersonal disrespect/rudeness)</td>
<td align="left" valign="top">Form of intolerance (moral disrespect)</td>
</tr>
<tr>
<td align="left" valign="top">Name-calling/insults: &#8216;you&#8217;re a bigoted cunt,&#8217; &#8216;his moronic views&#8217;<break/>Vulgarity: &#8216;fucking bullshit,&#8217; &#8216;what the fuck,&#8217; &#8216;crap&#8217;<break/>Pejorative remark about how one communicates: &#8216;Quit crying over the spilt milk of &#8230;,&#8217; &#8216;what a stupid argument&#8217;</td>
<td align="left" valign="top">Equation of abortion to intolerable extreme ideologies including genocide, the Nazi holocaust, slavery<break/>Demonisation: &#8216;They are devils,&#8217; &#8216;evil,&#8217; &#8216;satanic&#8217;<break/>Dehumanisation: &#8216;These pest/rats/roaches would be exterminated&#8217;<break/>Absolutism of one viewpoint as a generally and reciprocally binding morality: e.g., women/foetus have a right to freedom/life, it is immoral to infringe on it.<break/>Calling for violence, threats: e.g., saying that people should be punched/killed, buildings should be vandalised/arsoned</td>
</tr>
<tr>
<td colspan="2"><hr/></td>
</tr>
</table>
</table-wrap>
<p>To build incivility and intolerance classifiers, the authors manually coded 6,000 tweets from the American dataset and 5,700 tweets from the Irish dataset to create a list of uncivil and intolerant word stems and expressions. We then checked the performance (i.e., accuracy, precision, recall) of the classifiers and added more expressions to enhance the performance. The full manual labelling of these training samples was conducted by one coder, but multiple coders actively partook in the process of coding scheme development and inter-coder reliability tests. The coding scheme for manual qualitative labelling achieves substantial agreements in terms of both Cohen&#8217;s <italic>kappa</italic> and percentage agreement. The abortion issue position has 91.3% agreement and 0.83 <italic>kappa</italic>. The incivility variable has 93% agreement and 0.76 <italic>kappa</italic>, and intolerance has 94% agreement and 0.74 <italic>kappa</italic>. Percentage agreement was used to complement the overly conservative measure of beyond-chance agreement of Cohen&#8217;s <italic>kappa</italic> (<xref ref-type="bibr" rid="B26">Joyce, 2013</xref>).</p>
<p>The final version of the American incivility classifier achieved the mean accuracy of 85.8% with the mean precision of 91% and mean recall of 73.9%. The American intolerance classifier yielded the mean accuracy of 86.5% with the mean precision of 82% and the mean recall of 67.8%. The mean accuracy of the Irish incivility classifier was 93.7% with the mean precision of 86% and mean recall of 61.8%. The mean performance of the Irish intolerance classifier was 98.3% accuracy, 99.2% precision, and 70.8% recall. Recall performances were lower than accuracy or precision as it was difficult to identify every creative and nuanced form of incivility and intolerance in big data. For instance, studies have discussed the difficulty of automatic sarcasm detection on Twitter: sarcasm alone can cause a 50% drop in accuracy when automatically detecting sentiment (<xref ref-type="bibr" rid="B45">Sykora et al., 2020</xref>). Minimising false-positives provides a reliable standard for assessing the performance of natural language classifiers (<xref ref-type="bibr" rid="B32">Muddiman et al., 2019</xref>). The performance of our lexicon-based classifier matches the performance of ML classifiers such as in Theocharis et al. (<xref ref-type="bibr" rid="B46">2020</xref>).</p>
</sec>
<sec>
<title>6.2.2. Classifying abortion issue position</title>
<p>We coded the abortion issue positions of Twitter users into pro-choice, pro-life, and unidentifiable (either ambiguous or neutral to abortion). Since our American and Irish datasets were collected through different methods, two different tactics were used.</p>
<p>For the American dataset, we classified abortion issue positions using the tf-idf<xref ref-type="fn" rid="n5">5</xref> scores of the top 5,000 predictive n-grams<xref ref-type="fn" rid="n6">6</xref> for pro-choice and pro-life tweets. The training set for the American abortion position classification contained 3,000 tweets and the testing set contained 708 tweets. We then extracted n-grams with the top 2,500 tf-idf values from each pro-choice and pro-life position. Many n-grams contained the same tf-idf values, and therefore the top n-grams with the top 2,500 tf-idf values were more than 2,500: 2,803 pro-choice n-grams and 5,178 pro-life n-grams. Most of these top n-grams were bigrams and trigrams.</p>
<p>We classified American tweets based on the pro-life and pro-choice n-grams. When a tweet included pro-life/pro-choice n-grams only, the tweet was classified as a pro-life/pro-choice tweet. If a tweet contained both pro-life and pro-choice n-grams, the tweet was classified as position neutral or ambiguous. If a tweet did not contain any pro-life or pro-choice n-grams, it was also classified as position neutral or ambiguous. When evaluated on the testing set, the accuracy was 71.8%;higher than the baseline accuracy and as high as Sharma et al. (<xref ref-type="bibr" rid="B41">2017</xref>)&#8217;s ML ideology classifier.</p>
<p>Since the Irish dataset (<xref ref-type="bibr" rid="B30">Littman, 2018</xref>) was collected through 53 key hashtags with clear ideological indications, we classified the Irish dataset into pro-life, pro-choice, and unidentifiable positions following the ideological hashtags. If a tweet contained only pro-choice (or pro-life) hashtags, it was labelled as pro-choice (or pro-life). If a tweet did not have either pro-choice or pro-life hashtags, it was coded as unidentifiable. If a tweet had both pro-choice and pro-life hashtags simultaneously, it was also coded as unidentifiable. We randomly sampled 1,000 tweets from the Irish dataset and compared the hashtag-based automatic classification and manual classification. The accuracy of the hashtag-based classifier is 84%. This accuracy is as high as the state-of-the-art ML classification of abortion ideologies in Sharma et al. (<xref ref-type="bibr" rid="B41">2017</xref>) therefore our simple and intuitive classification models can perform as accurately as the best accuracy of a more complex ML classifier.</p>
</sec>
<sec>
<title>6.2.3. Classifying gender and anonymity</title>
<p>We used an R package for Automatic Gender Recognition (AGR): <italic>Gender</italic> (<xref ref-type="bibr" rid="B34">Mullen, 2021</xref>). This package predicts gender by name from historical data, typically population data gathered by American Census and Social Security data (<xref ref-type="bibr" rid="B34">Mullen, 2021</xref>). <italic>Gender</italic> automatically allocated Twitter users in our datasets into three nominal categories: women, men, N/A (gender unidentifiable; e.g., anonyms like &#8216;Old Glory&#8217;).</p>
<p>We treat gender as a control variable to our regression model since it can have confounding relationships with users&#8217; attitudes and behaviours to incivility and intolerance in online abortion discussions. First, gender can moderate the relationship between abortion issue positions, incivility, and intolerance. Studies suggest that reactionary voices against feminism and abortion rights often come from conservative men (<xref ref-type="bibr" rid="B35">Norris &amp; Inglehart, 2019</xref>). Wang and Silva (<xref ref-type="bibr" rid="B47">2018</xref>) find that in online abortion discussions, gender interacts with the broader opinion climate which then affects the relationship between exposure to incivility and increased negative emotions. Women feel more negative emotions when they see incivility directed at their side and when their view is in a disadvantageous position whereas men&#8217;s emotion or willingness to partake online are not affected by opinion climate (<xref ref-type="bibr" rid="B47">Wang &amp; Silva, 2018</xref>). Anonymity is associated with disinhibition and hence increased tendency to be uncivil without worrying about consequences (<xref ref-type="bibr" rid="B9">Coe et al., 2014</xref>).</p>
<p>Since our specific operationalisation of gender and anonymity carries limitations and hence impedes a statistical investigation with predictive power, we choose to have gender and anonymity as control variables. First and foremost, it is impossible to verify the &#8216;real&#8217; gender of the account holder with the plausible exception of blue-ticked, verified accounts. Second, the operationalisation of anonymity based on the first name has limitations. Someone could name their account with obvious pseudonyms such as John Doe, but would be classified as men and non-anonymous based on our chosen method. Nonetheless, predicting gender based on first names has been used in studies involving big online datasets (<xref ref-type="bibr" rid="B27">King &amp; Frederickson, 2021</xref>). By using this method, we are not assuming that all individuals are correctly gendered, but this provides insight into gender&#8217;s effects in aggregate.</p>
</sec>
<sec>
<title>6.2.4. Classifying discussion contexts</title>
<p>Our Twitter datasets collected through Twitter APIs contain meta-information about communication contexts, indicating whether a post was a tweet, retweet, quote, or reply. We treat tweet networks built through retweets and quotes as a proxy to a homogeneous discussion environment and treat replies as a proxy to a heterogeneous discussion environment. Our operationalisation is built on previous Twitter studies that have found that Twitter networks built through quotes and retweets contain agreements and homophily whereas the networks built through replies have more disagreements (<xref ref-type="bibr" rid="B16">Garimella et al., 2016</xref>). We do not assert that this simplified method correctly classifies every tweet, but it provides insights about the effect of discussion environments in aggregate.</p>
</sec>
<sec>
<title>6.2.5. Measuring Twitter coordination dynamics</title>
<p>We measured coordinated dynamics of incivility and intolerance by computing the Gini index for inequality in the distribution of tweets by users (<xref ref-type="bibr" rid="B46">Theocharis et al., 2020</xref>). Gini coefficients can capture coordination dynamics on Twitter: whether a small set of users are leading uncivil and intolerant communications. A higher Gini coefficient indicates greater levels of inequality, and this enables us to analyse whether a discussion is dominated by a small number of participants or whether it is open to a wide range of participants with similar levels of involvement. A Gini coefficient close to 1 means that most uncivil/intolerant tweets are posted by a single user, while a coefficient close to 0 would imply that all users are equally likely to post such content.</p>
<p>After classifying the tweets, logistic regression was employed to examine the relationship between independent variables (abortion issue positions, discussion contexts) and dependent variables (incivility and intolerance). Gender and anonymity are added to our model as control variables. We must also note the limitation in our regression analysis, like many others using Twitter data (e.g., <xref ref-type="bibr" rid="B46">Theocharis et al., 2020</xref>), where we treat each tweet as statistically independent. Although we do include a simplified approximation of the intricate relationships in a discussion, where the variable &#8216;discussion context&#8217; (i.e., original/initial tweet in a thread, reply, etc.) provides an approximate proxy for the relationship between an uncivil/intolerant tweet and the discussion context it is embedded in.</p>
</sec>
</sec>
</sec>
<sec>
<title>7. Results</title>
<p><xref ref-type="table" rid="T2">Table 2</xref> shows how the American and Irish Twitter datasets are constructed. Both datasets contain large portions of tweets posted by gender-unidentifiable users. One supposition might be that many users might have been concerned about voicing their views in the abortion issue publicly with their gender-identifiable names. This observation could prompt future studies to examine who uses anonymity and why.</p>
<table-wrap id="T2">
<label>Table 2</label>
<caption>
<p>Descriptive characteristics of the American and Irish Twitter datasets.</p>
</caption>
<table>
<tr>
<th colspan="6"><hr/></th>
</tr>
<tr>
<th align="left" valign="top" rowspan="3" colspan="2">Category</th>
<th align="center" valign="top" colspan="2">US</th>
<th align="center" valign="top" colspan="2">Ireland</th>
</tr>
<tr>
<td colspan="8"><hr/></td>
</tr>
<tr>
<th align="center" valign="top">Number</th>
<th align="center" valign="top">Per cent</th>
<th align="center" valign="top">Number</th>
<th align="center" valign="top">Per cent</th>
</tr>
<tr>
<td colspan="8"><hr/></td>
</tr>
<tr>
<td align="left" valign="top" rowspan="3">Gender</td>
<td align="left" valign="top">Women</td>
<td align="right" valign="top">1,722,144</td>
<td align="right" valign="top">28.40%</td>
<td align="right" valign="top">633,267</td>
<td align="right" valign="top">37.10%</td>
</tr>
<tr>
<td align="left" valign="top">Men</td>
<td align="right" valign="top">1,449,422</td>
<td align="right" valign="top">23.90%</td>
<td align="right" valign="top">415,413</td>
<td align="right" valign="top">24.30%</td>
</tr>
<tr>
<td align="left" valign="top">Unidentifiable</td>
<td align="right" valign="top">2,883,104</td>
<td align="right" valign="top">47.60%</td>
<td align="right" valign="top">659,299</td>
<td align="right" valign="top">38.60%</td>
</tr>
<tr>
<td align="left" valign="top" rowspan="3">Issue position</td>
<td align="left" valign="top">Pro-life</td>
<td align="right" valign="top">1,593,090</td>
<td align="right" valign="top">30.60%</td>
<td align="right" valign="top">198,906</td>
<td align="right" valign="top">11.60%</td>
</tr>
<tr>
<td align="left" valign="top">Pro-choice</td>
<td align="right" valign="top">1,852,511</td>
<td align="right" valign="top">26.30%</td>
<td align="right" valign="top">1,064,159</td>
<td align="right" valign="top">62.30%</td>
</tr>
<tr>
<td align="left" valign="top">Unidentifiable</td>
<td align="right" valign="top">2,609,069</td>
<td align="right" valign="top">43.10%</td>
<td align="right" valign="top">444,914</td>
<td align="right" valign="top">26%</td>
</tr>
<tr>
<td align="left" valign="top" rowspan="4">Tweet affordance</td>
<td align="left" valign="top">Original tweet</td>
<td align="right" valign="top">384,013</td>
<td align="right" valign="top">6.30%</td>
<td align="right" valign="top">189,505</td>
<td align="right" valign="top">11.10%</td>
</tr>
<tr>
<td align="left" valign="top">Retweet</td>
<td align="right" valign="top">4,415,076</td>
<td align="right" valign="top">72.90%</td>
<td align="right" valign="top">1,121,611</td>
<td align="right" valign="top">65.70%</td>
</tr>
<tr>
<td align="left" valign="top">Quote</td>
<td align="right" valign="top">509,140</td>
<td align="right" valign="top">8.40%</td>
<td align="right" valign="top">341,506</td>
<td align="right" valign="top">20%</td>
</tr>
<tr>
<td align="left" valign="top">Reply</td>
<td align="right" valign="top">746,441</td>
<td align="right" valign="top">12.30%</td>
<td align="right" valign="top">55,357</td>
<td align="right" valign="top">3.20%</td>
</tr>
<tr>
<td align="left" valign="top" rowspan="2">Discussion context</td>
<td align="left" valign="top">Homogeneous</td>
<td align="right" valign="top">5,091,400</td>
<td align="right" valign="top">81.30%</td>
<td align="right" valign="top">1,463,117</td>
<td align="right" valign="top">85.70%</td>
</tr>
<tr>
<td align="left" valign="top">Heterogeneous</td>
<td align="right" valign="top">746,441</td>
<td align="right" valign="top">12.30%</td>
<td align="right" valign="top">55,357</td>
<td align="right" valign="top">3.20%</td>
</tr>
<tr>
<td align="left" valign="top" colspan="2">Incivility</td>
<td align="right" valign="top">929,032</td>
<td align="right" valign="top">15.30%</td>
<td align="right" valign="top">153,028</td>
<td align="right" valign="top">9%</td>
</tr>
<tr>
<td align="left" valign="top" colspan="2">Intolerance</td>
<td align="right" valign="top">606,542</td>
<td align="right" valign="top">10%</td>
<td align="right" valign="top">29,469</td>
<td align="right" valign="top">1.70%</td>
</tr>
</table>
</table-wrap>
<p>Some notable differences between the two are: (1) the Irish dataset contains a larger portion of tweets by women than the American dataset. (2) The Irish dataset is predominantly pro-choice whereas the American data has a near-equal mix of pro- and anti-abortion tweets. (3) The Irish data consists of retweets followed by quotes and original tweets whereas American data consists of retweets followed by replies and quotes. Lastly, (4) the American dataset contains more incivility and intolerance than the Irish data (H1), which could be explained by the radical and violent history of abortion activism in the US (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>; <xref ref-type="bibr" rid="B24">Hunter, 1994</xref>).</p>
<p>Overall, both countries&#8217; Twitter abortion discourse involves a small amount of uncivil and intolerant communications. 15.3% and 10% of American tweets and 9% and 1.7% of Irish tweets contain some form of incivility and intolerance (RQ1). The occurrences of incivility and intolerance in our datasets are not particularly higher than the volume of incivility and intolerance found in other online political public spheres (<xref ref-type="bibr" rid="B9">Coe et al., 2014</xref>; <xref ref-type="bibr" rid="B38">Rossini, 2020</xref>; <xref ref-type="bibr" rid="B46">Theocharis et al., 2020</xref>). This indicates that the impact of incivility and intolerance is felt more strongly than their quantitative shares of the total abortion discourse (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>).</p>
<p>We calculated Pearson&#8217;s correlation coefficients (r) between incivility and intolerance. The coefficient between incivility and intolerance is 0.22 in the American data and 0.09 in the Irish data, showing positive linear trends. However, the sizes of both coefficients are weak.</p>
<p><xref ref-type="table" rid="T3">Table 3</xref> below shows the distribution of uncivil and intolerant tweets in the American and Irish datasets. To investigate the reach of the uncivil and intolerant tweets, we calculated the mean, median, and standard deviations (SDs) of the amount of public engagements the uncivil and/or intolerant tweets generate.</p>
<table-wrap id="T3">
<label>Table 3</label>
<caption>
<p>Number of tweets in the civility-tolerance matrix and the means, medians, and SDs of the number of retweets the tweets received.</p>
</caption>
<table>
<tr>
<th align="left" valign="top">US</th>
<th align="left" valign="top">Civility</th>
<th align="left" valign="top">Incivility</th>
</tr>
<tr>
<td colspan="3"><hr/></td>
</tr>
<tr>
<td align="left" valign="top"><bold>Tolerance</bold></td>
<td align="left" valign="top">4,754,878<break/>(mean: 16257.55, median: 559, SD: 32649.92)</td>
<td align="left" valign="top">693,250<break/>(mean: 2064.442, median: 192, SD: 4219.13)</td>
</tr>
<tr>
<td align="left" valign="top"><bold>Intolerance</bold></td>
<td align="left" valign="top">370,760<break/>(mean: 5425.39, median: 910, SD: 7916.27)</td>
<td align="left" valign="top">235,782<break/>(mean: 7360.46, median: 1372, SD: 11575.41)</td>
</tr>
<tr>
<td align="left" valign="top" colspan="3"><bold>Ireland</bold></td>
</tr>
<tr>
<td colspan="3"><hr/></td>
</tr>
<tr>
<td align="left" valign="top"><bold>Tolerance</bold></td>
<td align="left" valign="top">1,534,078<break/>(mean: 701.05, median: 26, SD: 2477.87)</td>
<td align="left" valign="top">144,432<break/>(mean: 2341.51, median: 124, SD: 4854.19)</td>
</tr>
<tr>
<td align="left" valign="top"><bold>Intolerance</bold></td>
<td align="left" valign="top">20,873<break/>(mean: 327.67, median: 56, SD: 492.82)</td>
<td align="left" valign="top">8,596<break/>(mean: 324.18, median: 84, SD: 427.14)</td>
</tr>
</table>
</table-wrap>
<p>Different types of tweets got more engagements in American and Irish Twitterspheres. In the American dataset, tweets that do not contain incivility nor intolerance earn the most retweets on average. As for the median of retweets, uncivil and intolerant tweets got most public engagements. In the Irish case, uncivil tweets earned the most retweets both in terms of their mean and median values. Considering median measures is important here, since distributions of retweets can be skewed by a small proportion of extremely popular tweets and a large portion of tweets that earned zero retweets. This indicates that although incivility and intolerance are uncommon behaviours, they still provoke huge volumes of engagements thereby influencing the atmosphere and direction of online public debates as well as the wider discussions on news media and elite discourse outside of Twitter.</p>
<p>To test other research hypotheses, a logistic regression analysis examines the relationship between diverse demographic characteristics and communicative contexts, incivility, and intolerance. Log odds are transformed to odds ratios to facilitate interpretation and 95% confidence intervals (CIs) are reported in addition to p-values. Reporting CIs is especially important to complement the error-prone nature of p-values in big data studies (<xref ref-type="bibr" rid="B29">Lin et al., 2013</xref>). Results of the regression analysis are in <xref ref-type="table" rid="T4">Table 4</xref>.</p>
<table-wrap id="T4">
<label>Table 4</label>
<caption>
<p>Odds ratio and 95% confidence intervals of incivility and intolerance in American and Irish abortion discourse on Twitter.</p>
</caption>
<table>
<tr>
<th colspan="9"><hr/></th>
</tr>
<tr>
<th align="left" valign="top" rowspan="5"></th>
<th align="center" valign="top" colspan="4">US</th>
<th align="center" valign="top" colspan="4">Ireland</th>
</tr>
<tr>
<td colspan="8"><hr/></td>
</tr>
<tr>
<th align="center" valign="top" colspan="2">Incivility</th>
<th align="center" valign="top" colspan="2">Intolerance</th>
<th align="center" valign="top" colspan="2">Incivility</th>
<th align="center" valign="top" colspan="4">Intolerance</th>
</tr>
<tr>
<td colspan="8"><hr/></td>
</tr>
<tr>
<th align="center" valign="top">OR</th>
<th align="center" valign="top">CI [2.5%, 97.5%]</th>
<th align="center" valign="top">OR</th>
<th align="center" valign="top">CI [2.5%, 97.5%]</th>
<th align="center" valign="top">OR</th>
<th align="center" valign="top">CI [2.5%, 97.5%]</th>
<th align="center" valign="top">OR</th>
<th align="center" valign="top">CI [2.5%, 97.5%]</th>
</tr>
<tr>
<td colspan="9"><hr/></td>
</tr>
<tr>
<td align="left" valign="top">(Intercept)</td>
<td align="right" valign="top">0.11***</td>
<td align="right" valign="top">[0.11, 0.11]</td>
<td align="right" valign="top">0.03***</td>
<td align="right" valign="top">[0.03, 0.03]</td>
<td align="right" valign="top">0.09***</td>
<td align="right" valign="top">[0.09, 0.09]</td>
<td align="right" valign="top">0.00***</td>
<td align="right" valign="top">[0.00, 0.01]</td>
</tr>
<tr>
<td align="left" valign="top">Pro-life</td>
<td align="right" valign="top">2.19***</td>
<td align="right" valign="top">[2.18, 2.21]</td>
<td align="right" valign="top">8.53***</td>
<td align="right" valign="top">[8.45, 8.61]</td>
<td align="right" valign="top">0.89***</td>
<td align="right" valign="top">[0.87, 0.90]</td>
<td align="right" valign="top">12.59***</td>
<td align="right" valign="top">[12.21, 12.99]</td>
</tr>
<tr>
<td align="left" valign="top">Position unidentifiable</td>
<td align="right" valign="top">1.46***</td>
<td align="right" valign="top">[1.46, 1.47]</td>
<td align="right" valign="top">2.21***</td>
<td align="right" valign="top">[2.18, 2.23]</td>
<td align="right" valign="top">1.42***</td>
<td align="right" valign="top">[1.41, 1.44]</td>
<td align="right" valign="top">3.81***</td>
<td align="right" valign="top">[3.69, 3.94]</td>
</tr>
<tr>
<td align="left" valign="top">Heterogeneous environment</td>
<td align="right" valign="top">1.26***</td>
<td align="right" valign="top">[1.25, 1.26]</td>
<td align="right" valign="top">0.55***</td>
<td align="right" valign="top">[0.55, 0.56]</td>
<td align="right" valign="top">1.11***</td>
<td align="right" valign="top">[1.07, 1.14]</td>
<td align="right" valign="top">0.85***</td>
<td align="right" valign="top">[0.8, 0.91]</td>
</tr>
<tr>
<td align="left" valign="top">Gender men</td>
<td align="right" valign="top">1.06***</td>
<td align="right" valign="top">[1.05, 1.06]</td>
<td align="right" valign="top">1.09***</td>
<td align="right" valign="top">[1.08, 1.1]</td>
<td align="right" valign="top">1.11***</td>
<td align="right" valign="top">[1.1, 1.13]</td>
<td align="right" valign="top">1.4***</td>
<td align="right" valign="top">[1.36, 1.45]</td>
</tr>
<tr>
<td align="left" valign="top">Gender unidentifiable</td>
<td align="right" valign="top">1.07***</td>
<td align="right" valign="top">[1.07, 1.08]</td>
<td align="right" valign="top">1.05***</td>
<td align="right" valign="top">[1.04, 1.06]</td>
<td align="right" valign="top">1.1***</td>
<td align="right" valign="top">[1.09, 1.12]</td>
<td align="right" valign="top">1.35***</td>
<td align="right" valign="top">[1.32, 1.39]</td>
</tr>
</table>
<table-wrap-foot>
<fn><p>Signif. Codes: 0 &#8216;***&#8217;.</p></fn>
</table-wrap-foot>
</table-wrap>
<p>The results show that the pro-life issue position is strongly related to more intolerance in both countries. The odds of intolerant communications for pro-life tweets are 8.53 times higher in the American data and 12.59 times higher in the Irish dataset than pro-choice tweets. The relationship between incivility and abortion issue positions shows conflicting results. The odds of uncivil tweets for pro-life tweets are 2.19 times higher in the US, but 0.89 times lower in Ireland. This shows that in the American Twittersphere, pro-life tweets are more likely to be uncivil and intolerant, while in the Irish Twittersphere, pro-life tweets are more associated with intolerance whereas pro-choice tweets express more incivility (H2 supported).</p>
<p>Furthermore, our micro-level analysis shows that uncivil and intolerant Twitter communications are mobilised by a small number of &#8216;superparticipants&#8217; (<xref ref-type="bibr" rid="B18">Graham &amp; Wright, 2014</xref>) (H3 rejected). In the American dataset, the Gini coefficient for the tweet distribution is 0.6. The level of coordination is larger for uncivil and intolerant tweets: 0.85 for incivility and 0.92 for intolerance. In the American abortion discourse on Twitter, the top 1% of most active users are responsible for 27.1% of the total incivility and 39.5% of total intolerance. This pattern is also observed for our Irish dataset where the Gini coefficient for the tweet distribution is 0.73. The Gini coefficient for incivility is 0.88 and the coefficient for intolerance is 0.97. The top 1% of the most active Irish users express 41% of the total incivility and almost entire intolerance in the dataset. <xref ref-type="fig" rid="F1">Figure 1</xref> visualises the Lorenz curves for the distribution of total tweets, incivility, and intolerance in the two datasets.</p>
<fig id="F1">
<label>Figure 1</label>
<caption>
<p>Lorenz curves for user coordination dynamics in American and Irish datasets.</p>
</caption>
<graphic xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="jdd-19-1-1413-g1.png"/>
</fig>
<p>The regression model also reveals a meso-level relationship between discussion environments and incivility and intolerance. The odds of incivility are 1.26 times higher in the American data and 1.11 times higher in the Irish dataset when a tweet is embedded in a heterogeneous environment where Twitter users are actively disagreeing with other tweets. In contrast, the odds of intolerance become 0.55 times lower in the American data and 0.85 times lower in the Irish data, meaning that intolerance is strongly associated with a homogeneous environment, lacking disagreements and inter-group engagements (H4 confirmed).</p>
</sec>
<sec>
<title>8. Concluding remarks and future directions</title>
<p>We explore Twitter discussions as a facilitator of abortion issue public, evaluating the public capabilities for deliberating highly divisive culture war issues like abortion (<xref ref-type="bibr" rid="B24">Hunter, 1994</xref>) while observing to the normative deliberative criteria of mutual respect. We operationalise mutual respect with two concepts: civility (interpersonal politeness) and tolerance (moral-political respect). Studying American and Irish cases together provides findings on the potential similarities and transferability of the results.</p>
<p>Despite choosing an emotive and polarised issue like abortion, the Twitter discussions in the US and Ireland are mostly civil and tolerant. This contrasts with perceived culture wars and antipathy in their abortion debates (<xref ref-type="bibr" rid="B5">Brownstein, 2021</xref>) suggesting that the impact of incivility and intolerance outstrips their actual representation in the discourse (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>). However, we find that uncivil and intolerant tweets generate more public engagements, potentially influencing a wider discussion.</p>
<p>We also find that American abortion discussions contain more incivility and significantly more intolerance than Irish discussions, potentially due to America&#8217;s history of violent abortion activism (<xref ref-type="bibr" rid="B14">Ferree et al., 2002</xref>) and Ireland&#8217;s experience with deliberative minipublics over contentious issues e.g., same-sex marriage (<xref ref-type="bibr" rid="B44">Suiter, 2018</xref>). This invites further research into whether the institutional uses of deliberative politics increase deliberative qualities of online discussions globally.</p>
<p>The regression analysis reveals that uncivil and intolerant communications on Twitter are associated with different demographic and discussion contexts. The micro-level analysis identified intolerance as a distinctively pro-life behaviour in both American and Irish Twitterspheres, supporting existing literature linking uncivil society with reactionary right-wing discourse (<xref ref-type="bibr" rid="B35">Norris &amp; Inglehart, 2019</xref>).</p>
<p>In our meso-level analysis, we find that incivility prevails in heterogeneous environments whereas intolerance occurs within self-selected echo chambers (<xref ref-type="bibr" rid="B38">Rossini, 2020</xref>). This indicates the necessity to intervene in spiralling intolerant echo chambers.</p>
<p>Another noteworthy micro-level finding is the coordinated dynamics of uncivil and intolerant behaviours (<xref ref-type="bibr" rid="B46">Theocharis et al., 2020</xref>). Uncivil and intolerant discourse on Twitter is largely expressed and circulated by a small set of &#8216;superparticipants&#8217; (<xref ref-type="bibr" rid="B18">Graham &amp; Wright, 2014</xref>). The overrepresentation of hyperpartisan voices on social media leads to a distorted understanding of how polarised and extreme the opposingside is, making politically moderate citizens leave the online discourse (<xref ref-type="bibr" rid="B1">Bail, 2021</xref>) creating subsequent inequalities in policy outcomes where hyper-partisan voices are strongly represented and heard more by politicians and policymakers in digital spaces (<xref ref-type="bibr" rid="B28">Kleiner, 2018</xref>). It is crucial for online deliberation to curtail the capabilities of these superparticipants distorting and radicalising the overall online political discourse. The development of a meaningful typology of hyperactive users to generate a deeper understanding of who these people are and how they operate is needed.</p>
</sec>
</body>
<back>
<fn-group>
<fn id="n1"><p>To respectfully represent both sides, we have chosen to use the collective labels that each side prefers to self-identify with (<xref ref-type="bibr" rid="B49">YouGov, 2022</xref>). Labels such as pro-abortion and anti-abortion are criticised by some pro-choice and pro-life groups for being reductive (<xref ref-type="bibr" rid="B36">Pew, 2008</xref>; <xref ref-type="bibr" rid="B40">Save the Storks, 2019</xref>).</p></fn>
<fn id="n2"><p>Echo chambers refer to environments where people only encounter and engage with information and opinions that are similar to their own. Our understanding focuses on self-imposed echo chambers (users actively choose to consume certain information) than algorithmically-driven understanding of the phenomenon (<xref ref-type="bibr" rid="B6">Bruns, 2019</xref>).</p></fn>
<fn id="n3"><p>Data rehydration is the process of importing data so it can be readily accessed and used. This is done using tools, e.g., Twarc2 (<ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://twarc-project.readthedocs.io/en/latest/twarc2_en_us/#hydrate">https://twarc-project.readthedocs.io/en/latest/twarc2_en_us/#hydrate</ext-link>). For official information, see Twitter developer at <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/overview">https://developer.twitter.com/en/docs/twitter-api/v1/tweets/post-and-engage/overview</ext-link>.</p></fn>
<fn id="n4"><p>Our coding scheme is available here: <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://docs.google.com/document/d/1EUgPQn3l5wsHiur-79KfiulWYv0FdrwLLt9cHnzzUbw/edit?usp=sharing">https://docs.google.com/document/d/1EUgPQn3l5wsHiur-79KfiulWYv0FdrwLLt9cHnzzUbw/edit?usp=sharing</ext-link>.</p></fn>
<fn id="n5"><p>Tf-idf stands for Term frequency and Inverse Document Frequency matrix. Tf-idf weighs common words and rare words, calculating the importance of a word to a specific document (<xref ref-type="bibr" rid="B42">Silge &amp; Robinson, 2017</xref>).</p></fn>
<fn id="n6"><p>In natural language processing, an n-gram is a parsing of a text into a number of sequences of words or characters. For sequences of words, unigrams (n = 1) that can be generated from a sentence &#8216;How are you&#8217; are &#8216;how,&#8217; &#8216;are,&#8217; &#8216;you&#8217;; possible bigrams (n = 2) are &#8216;how are&#8217; and &#8216;are you.&#8217;</p></fn>
</fn-group>
<sec>
<title>Open Source</title>
<p>The R code and dictionary text files are freely available via the Open Science Framework (OSF) and can be accessed by following this link: <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://osf.io/u6gw2/">https://osf.io/u6gw2/</ext-link>.</p>
</sec>
<sec>
<title>Competing Interests</title>
<p>The authors have no competing interests to declare.</p>
</sec>
<sec>
<title>Publishers Note</title>
<p><xref ref-type="table" rid="T3">Table 3</xref> was updated on 20th October 2023 to re-insert one row that was missing from the original publication. This row specifies that the bottom two rows are regarding the Irish case.</p>
</sec>
<ref-list>
<ref id="B1"><label>1</label><mixed-citation publication-type="book"><string-name><surname>Bail</surname>, <given-names>C.</given-names></string-name> (<year>2021</year>). <source>Breaking the social media prism</source>. <publisher-name>Princeton University Press</publisher-name>. DOI: <pub-id pub-id-type="doi">10.1515/9780691216508</pub-id></mixed-citation></ref>
<ref id="B2"><label>2</label><mixed-citation publication-type="journal"><string-name><surname>Barbera</surname>, <given-names>P.</given-names></string-name>, &amp; <string-name><surname>Rivero</surname>, <given-names>G.</given-names></string-name> (<year>2015</year>). <article-title>Understanding the political representativeness of twitter users</article-title>. <source>Social Science Computer Review</source>, <volume>33</volume>(<issue>6</issue>), <fpage>712</fpage>&#8211;<lpage>729</lpage>. DOI: <pub-id pub-id-type="doi">10.1177/0894439314558836</pub-id></mixed-citation></ref>
<ref id="B3"><label>3</label><mixed-citation publication-type="confproc"><string-name><surname>Bender</surname>, <given-names>E. M.</given-names></string-name>, <string-name><surname>Gebru</surname>, <given-names>T.</given-names></string-name>, <string-name><surname>McMillan-Major</surname>, <given-names>A.</given-names></string-name>, &amp; <string-name><surname>Shmitchell</surname>, <given-names>S.</given-names></string-name> (<year>2021</year>). <article-title>On the dangers of stochastic parrots: Can language models be too big?</article-title> In <conf-name>Proceedings of the 2021 acm conference on fairness, accountability, and transparency</conf-name>, <fpage>610</fpage>&#8211;<lpage>623</lpage>. DOI: <pub-id pub-id-type="doi">10.1145/3442188.3445922</pub-id></mixed-citation></ref>
<ref id="B4"><label>4</label><mixed-citation publication-type="book"><string-name><surname>Berry</surname>, <given-names>J. M.</given-names></string-name>, &amp; <string-name><surname>Sobieraj</surname>, <given-names>S.</given-names></string-name> (<year>2013</year>). <source>The outrage industry: Political opinion media and the new incivility</source>. <publisher-name>Oxford University Press</publisher-name>.</mixed-citation></ref>
<ref id="B5"><label>5</label><mixed-citation publication-type="webpage"><string-name><surname>Brownstein</surname>, <given-names>R.</given-names></string-name> (<year>2021</year>, <day>25</day> <month>May</month>). <article-title>A culture war with real consequences is coming</article-title>. <source>CNN</source>. <uri>https://edition.cnn.com/2021/05/25/politics/abortion-mississippi-supreme-court-culture-war/index.html</uri></mixed-citation></ref>
<ref id="B6"><label>6</label><mixed-citation publication-type="journal"><string-name><surname>Bruns</surname>, <given-names>A.</given-names></string-name> (<year>2019</year>). <article-title>It&#8217;s not the technology, stupid: How the &#8216;Echo Chamber&#8217; and &#8216;Filter Bubble&#8217; metaphors have failed us</article-title>. <source>Paper presented at the 2019 IAMCR conference</source>, <fpage>1</fpage>&#8211;<lpage>12</lpage>.</mixed-citation></ref>
<ref id="B7"><label>7</label><mixed-citation publication-type="book"><string-name><surname>Bruns</surname>, <given-names>A.</given-names></string-name>, &amp; <string-name><surname>Highfield</surname>, <given-names>T.</given-names></string-name> (<year>2015</year>). <chapter-title>Is Habermas on Twitter? Social media and the public sphere</chapter-title>. In <source>the Routledge companion to social media and politics</source>, <fpage>56</fpage>&#8211;<lpage>73</lpage>. <publisher-name>Routledge</publisher-name>. DOI: <pub-id pub-id-type="doi">10.4324/9781315716299-5</pub-id></mixed-citation></ref>
<ref id="B8"><label>8</label><mixed-citation publication-type="confproc"><string-name><surname>Chen</surname>, <given-names>C.</given-names></string-name>, <string-name><surname>Zhang</surname>, <given-names>J.</given-names></string-name>, <string-name><surname>Chen</surname>, <given-names>X.</given-names></string-name>, <string-name><surname>Xiang</surname>, <given-names>Y.</given-names></string-name>, &amp; <string-name><surname>Zhou</surname>, <given-names>W.</given-names></string-name> (<year>2015</year>). <article-title>6 million spam tweets: A large ground truth for timely Twitter spam detection</article-title>. In <conf-name>2015 ieee international conference on communications (icc)</conf-name>, <fpage>7065</fpage>&#8211;<lpage>7070</lpage>. DOI: <pub-id pub-id-type="doi">10.1109/ICC.2015.7249453</pub-id></mixed-citation></ref>
<ref id="B9"><label>9</label><mixed-citation publication-type="journal"><string-name><surname>Coe</surname>, <given-names>K.</given-names></string-name>, <string-name><surname>Kenski</surname>, <given-names>K.</given-names></string-name>, &amp; <string-name><surname>Rains</surname>, <given-names>S. A.</given-names></string-name> (<year>2014</year>). <article-title>Online and uncivil? Patterns and determinants of incivility in newspaper website comments</article-title>. <source>Journal of Communication</source>, <volume>64</volume>(<issue>4</issue>), <fpage>658</fpage>&#8211;<lpage>679</lpage>. DOI: <pub-id pub-id-type="doi">10.1111/jcom.12104</pub-id></mixed-citation></ref>
<ref id="B10"><label>10</label><mixed-citation publication-type="book"><string-name><surname>Cohen</surname>, <given-names>J.</given-names></string-name> (<year>1989</year>). <chapter-title>&#8220;Deliberation and Democratic Legitimacy.&#8221;</chapter-title> In <string-name><given-names>Alan</given-names> <surname>Hamlin</surname></string-name> and <string-name><given-names>Philip</given-names> <surname>Pettit</surname></string-name> (eds), <source>The Good Polity</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>Blackwell</publisher-name>, <fpage>17</fpage>&#8211;<lpage>34</lpage>.</mixed-citation></ref>
<ref id="B11"><label>11</label><mixed-citation publication-type="book"><string-name><surname>Dahlgren</surname>, <given-names>P.</given-names></string-name> (<year>2009</year>). <source>Media and political engagement</source>. <publisher-loc>Cambridge</publisher-loc>: <publisher-name>Cambridge University Press</publisher-name>.</mixed-citation></ref>
<ref id="B12"><label>12</label><mixed-citation publication-type="journal"><string-name><surname>Elayan</surname>, <given-names>S.</given-names></string-name>, <string-name><surname>Sykora</surname>, <given-names>M.</given-names></string-name>, &amp; <string-name><surname>Jackson</surname>, <given-names>T.</given-names></string-name> (<year>2020</year>). <article-title>&#8220;His Tweets Speak for Themselves&#8221;: An Analysis of Donald Trump&#8217;s Twitter Behavior</article-title>. <source>The International Journal of Interdisciplinary Civic and Political Studies</source>, <volume>15</volume>(<issue>1</issue>), <fpage>11</fpage>. DOI: <pub-id pub-id-type="doi">10.18848/2327-0071/CGP/v15i01/11-35</pub-id></mixed-citation></ref>
<ref id="B13"><label>13</label><mixed-citation publication-type="journal"><string-name><surname>Escobar</surname>, <given-names>O.</given-names></string-name>, &amp; <string-name><surname>Elstub</surname>, <given-names>S.</given-names></string-name> (<year>2017</year>). <article-title>Forms of mini-publics: An introduction to deliberative innovations in democratic practice</article-title>. <source>New Democracy. Research and Development Note</source>, <fpage>4</fpage>.</mixed-citation></ref>
<ref id="B14"><label>14</label><mixed-citation publication-type="book"><string-name><surname>Ferree</surname>, <given-names>M. M.</given-names></string-name>, <string-name><surname>Gamson</surname>, <given-names>W. A.</given-names></string-name>, &amp; <string-name><surname>Rucht</surname>, <given-names>D.</given-names></string-name> (<year>2002</year>). <source>Shaping abortion discourse: Democracy and the public sphere in Germany and the United States</source>. <publisher-name>Cambridge University Press</publisher-name>. DOI: <pub-id pub-id-type="doi">10.1017/CBO9780511613685</pub-id></mixed-citation></ref>
<ref id="B15"><label>15</label><mixed-citation publication-type="journal"><string-name><surname>Field</surname>, <given-names>L.</given-names></string-name> (<year>2018</year>). <article-title>The abortion referendum of 2018 and a timeline of abortion politics in Ireland to date</article-title>. <source>Irish Political Studies</source>, <volume>33</volume>(<issue>4</issue>), <fpage>608</fpage>&#8211;<lpage>628</lpage>. DOI: <pub-id pub-id-type="doi">10.1080/07907184.2018.1500461</pub-id></mixed-citation></ref>
<ref id="B16"><label>16</label><mixed-citation publication-type="confproc"><string-name><surname>Garimella</surname>, <given-names>K.</given-names></string-name>, <string-name><surname>Weber</surname>, <given-names>I.</given-names></string-name>, &amp; De <string-name><surname>Choudhury</surname>, <given-names>M.</given-names></string-name> (<year>2016</year>). <article-title>Quote RTs on Twitter: Usage of the New Feature for Political Discourse</article-title>. In <conf-name>Proceedings of the 8th acm conference on web science</conf-name>, <fpage>200</fpage>&#8211;<lpage>204</lpage>. DOI: <pub-id pub-id-type="doi">10.1145/2908131.2908170</pub-id></mixed-citation></ref>
<ref id="B17"><label>17</label><mixed-citation publication-type="journal"><string-name><surname>Garrett</surname>, <given-names>R. K.</given-names></string-name> (<year>2017</year>). <article-title>The &#8220;echo chamber&#8221; distraction: Disinformation campaigns are the problem, not audience fragmentation</article-title>. <source>Journal of Applied Research in Memory and Cognition</source>, <volume>6</volume>(<issue>4</issue>), <fpage>370</fpage>&#8211;<lpage>376</lpage>. DOI: <pub-id pub-id-type="doi">10.1016/j.jarmac.2017.09.011</pub-id></mixed-citation></ref>
<ref id="B18"><label>18</label><mixed-citation publication-type="journal"><string-name><surname>Graham</surname>, <given-names>T.</given-names></string-name>, &amp; <string-name><surname>Wright</surname>, <given-names>S.</given-names></string-name> (<year>2014</year>). <article-title>Discursive Equality and Everyday Talk Online: The Impact of &#8220;Superparticipants&#8221;</article-title>. <source>Journal of Computer-Mediated Communication</source>, <volume>19</volume>(<issue>3</issue>), <fpage>625</fpage>&#8211;<lpage>642</lpage>. DOI: <pub-id pub-id-type="doi">10.1111/jcc4.12016</pub-id></mixed-citation></ref>
<ref id="B19"><label>19</label><mixed-citation publication-type="book"><string-name><surname>Gutmann</surname>, <given-names>A.</given-names></string-name>, &amp; <string-name><surname>Thompson</surname>, <given-names>D.</given-names></string-name> (<year>1996</year>). <source>Democracy and disagreement</source>. <publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>Belknap Press</publisher-name>.</mixed-citation></ref>
<ref id="B20"><label>20</label><mixed-citation publication-type="book"><string-name><surname>Habermas</surname>, <given-names>J.</given-names></string-name> (<year>1990</year>). <source>Moral consciousness and communicative action</source>. <publisher-name>MIT press</publisher-name>.</mixed-citation></ref>
<ref id="B21"><label>21</label><mixed-citation publication-type="book"><string-name><surname>Habermas</surname>, <given-names>J.</given-names></string-name> (<year>1996</year>). <source>Between facts and norms</source> (<string-name><given-names>W.</given-names> <surname>Rehg</surname></string-name>, Trans.). <publisher-name>The MIT Press</publisher-name>. DOI: <pub-id pub-id-type="doi">10.7551/mitpress/1564.001.0001</pub-id></mixed-citation></ref>
<ref id="B22"><label>22</label><mixed-citation publication-type="journal"><string-name><surname>Habermas</surname>, <given-names>J.</given-names></string-name> (<year>2003</year>). <article-title>Intolerance and discrimination</article-title>. <source>International Journal of Constitutional Law</source>, <volume>1</volume>(<issue>1</issue>), <fpage>2</fpage>&#8211;<lpage>12</lpage>. DOI: <pub-id pub-id-type="doi">10.1093/icon/1.1.2</pub-id></mixed-citation></ref>
<ref id="B23"><label>23</label><mixed-citation publication-type="journal"><string-name><surname>Habermas</surname>, <given-names>J.</given-names></string-name> (<year>2006</year>). <article-title>Political communication in media society: Does democracy still enjoy an epistemic dimension? The impact of normative theory on empirical research</article-title>. <source>Communication Theory</source>, <volume>16</volume>(<issue>4</issue>), <fpage>411</fpage>&#8211;<lpage>426</lpage>. DOI: <pub-id pub-id-type="doi">10.1111/j.1468-2885.2006.00280.x</pub-id></mixed-citation></ref>
<ref id="B24"><label>24</label><mixed-citation publication-type="book"><string-name><surname>Hunter</surname>, <given-names>J. D.</given-names></string-name> (<year>1994</year>). <source>Before the shooting begins</source>. <publisher-name>Simon and Schuster</publisher-name>.</mixed-citation></ref>
<ref id="B25"><label>25</label><mixed-citation publication-type="journal"><string-name><surname>Jashinsky</surname>, <given-names>J.</given-names></string-name>, <string-name><surname>Burton</surname>, <given-names>S. H.</given-names></string-name>, <string-name><surname>Hanson</surname>, <given-names>C. L.</given-names></string-name>, <string-name><surname>West</surname>, <given-names>J.</given-names></string-name>, <string-name><surname>Giraud-Carrier</surname>, <given-names>C.</given-names></string-name>, <string-name><surname>Barnes</surname>, <given-names>M. D.</given-names></string-name>, &amp; <string-name><surname>Argyle</surname>, <given-names>T.</given-names></string-name> (<year>2014</year>). <article-title>Tracking suicide risk factors through Twitter in the US</article-title>. <source>Crisis</source>. DOI: <pub-id pub-id-type="doi">10.1027/0227-5910/a000234</pub-id></mixed-citation></ref>
<ref id="B26"><label>26</label><mixed-citation publication-type="journal"><string-name><surname>Joyce</surname>, <given-names>M.</given-names></string-name> (<year>2013</year>). <article-title>Picking the best intercoder reliability statistic for your digital activism content analysis</article-title>. <source>Digital Activism Research Project</source>, <volume>243</volume>(<issue>11</issue>).</mixed-citation></ref>
<ref id="B27"><label>27</label><mixed-citation publication-type="journal"><string-name><surname>King</surname>, <given-names>M. M.</given-names></string-name>, &amp; <string-name><surname>Frederickson</surname>, <given-names>M. E.</given-names></string-name> (<year>2021</year>). <article-title>The pandemic penalty: the gendered effects of COVID-19 on scientific productivity</article-title>. <source>Socius</source>, <volume>7</volume>, <elocation-id>23780231211006977</elocation-id>. DOI: <pub-id pub-id-type="doi">10.1177/23780231211006977</pub-id></mixed-citation></ref>
<ref id="B28"><label>28</label><mixed-citation publication-type="journal"><string-name><surname>Kleiner</surname>, <given-names>T.</given-names></string-name> (<year>2018</year>). <article-title>Public opinion polarisation and protest behaviour</article-title>. <source>European Journal of Political Research</source>, <volume>57</volume>(<issue>4</issue>), <fpage>941</fpage>&#8211;<lpage>962</lpage>. DOI: <pub-id pub-id-type="doi">10.1111/1475-6765.12260</pub-id></mixed-citation></ref>
<ref id="B29"><label>29</label><mixed-citation publication-type="journal"><string-name><surname>Lin</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Lucas</surname>, <given-names>H. C.</given-names>, <suffix>Jr.</suffix></string-name>, &amp; <string-name><surname>Shmueli</surname>, <given-names>G.</given-names></string-name> (<year>2013</year>). <article-title>Research commentary&#8212;too big to fail: large samples and the p-value problem</article-title>. <source>Information Systems Research</source>, <volume>24</volume>(<issue>4</issue>), <fpage>906</fpage>&#8211;<lpage>917</lpage>. DOI: <pub-id pub-id-type="doi">10.1287/isre.2013.0480</pub-id></mixed-citation></ref>
<ref id="B30"><label>30</label><mixed-citation publication-type="book"><string-name><surname>Littman</surname>, <given-names>J.</given-names></string-name> (<year>2018</year>). <source>Ireland 8th Tweet Ids</source>. <publisher-name>Harvard Dataverse</publisher-name>, V1. DOI: <pub-id pub-id-type="doi">10.7910/DVN/PYCLPE</pub-id></mixed-citation></ref>
<ref id="B31"><label>31</label><mixed-citation publication-type="journal"><string-name><surname>Maia</surname>, <given-names>R.</given-names></string-name>, <string-name><surname>Cal</surname>, <given-names>D.</given-names></string-name>, <string-name><surname>Bargas</surname>, <given-names>J. K.</given-names></string-name>, <string-name><surname>Oliveira</surname>, <given-names>V. V.</given-names></string-name>, <string-name><surname>Rossini</surname>, <given-names>P. G.</given-names></string-name>, &amp; <string-name><surname>Sampaio</surname>, <given-names>R. C.</given-names></string-name> (<year>2017</year>). <article-title>Authority and deliberative moments: Assessing equality and inequality in deeply divided groups</article-title>. <source>Journal of Public Deliberation</source>, <volume>13</volume>(<issue>2</issue>). DOI: <pub-id pub-id-type="doi">10.16997/jdd.283</pub-id></mixed-citation></ref>
<ref id="B32"><label>32</label><mixed-citation publication-type="journal"><string-name><surname>Muddiman</surname>, <given-names>A.</given-names></string-name>, <string-name><surname>McGregor</surname>, <given-names>S. C.</given-names></string-name>, &amp; <string-name><surname>Stroud</surname>, <given-names>N. J.</given-names></string-name> (<year>2019</year>). <article-title>(Re) claiming our expertise: Parsing large text corpora with manually validated and organic dictionaries</article-title>. <source>Political Communication</source>, <volume>36</volume>(<issue>2</issue>), <fpage>214</fpage>&#8211;<lpage>226</lpage>. DOI: <pub-id pub-id-type="doi">10.1080/10584609.2018.1517843</pub-id></mixed-citation></ref>
<ref id="B33"><label>33</label><mixed-citation publication-type="journal"><string-name><surname>Mukhtar</surname>, <given-names>N.</given-names></string-name>, <string-name><surname>Khan</surname>, <given-names>M. A.</given-names></string-name>, &amp; <string-name><surname>Chiragh</surname>, <given-names>N.</given-names></string-name> (<year>2018</year>). <article-title>Lexicon-based approach outperforms Supervised Machine Learning approach for Urdu Sentiment Analysis in multiple domains</article-title>. <source>Telematics and Informatics</source>, <volume>35</volume>(<issue>8</issue>), <fpage>2173</fpage>&#8211;<lpage>2183</lpage>. DOI: <pub-id pub-id-type="doi">10.1016/j.tele.2018.08.003</pub-id></mixed-citation></ref>
<ref id="B34"><label>34</label><mixed-citation publication-type="webpage"><string-name><surname>Mullen</surname>, <given-names>L.</given-names></string-name> (<year>2021</year>). <source>gender: Predict gender from names using historical data</source>. r package version 0.5.4.1000. <uri>https://github.com/ropensci/gender/</uri>.</mixed-citation></ref>
<ref id="B35"><label>35</label><mixed-citation publication-type="book"><string-name><surname>Norris</surname>, <given-names>P.</given-names></string-name>, &amp; <string-name><surname>Inglehart</surname>, <given-names>R.</given-names></string-name> (<year>2019</year>). <source>Cultural backlash: Trump, Brexit, and authoritarian populism</source>. <publisher-name>Cambridge University Press</publisher-name>. DOI: <pub-id pub-id-type="doi">10.1017/9781108595841</pub-id></mixed-citation></ref>
<ref id="B36"><label>36</label><mixed-citation publication-type="webpage"><collab>Pew Research Centre</collab>. (<year>2008</year>). <source>Pro-Choice Does Not Mean Pro-Abortion: An Argument for Abortion Rights Featuring the Rev. Carlton Veazey</source>. <uri>https://www.pewresearch.org/religion/2008/09/30/pro-choice-does-not-mean-pro-abortion-an-argument-for-abortion-rights-featuring-the-rev-carlton-veazey/</uri></mixed-citation></ref>
<ref id="B37"><label>37</label><mixed-citation publication-type="journal"><string-name><surname>Rains</surname>, <given-names>S. A.</given-names></string-name>, <string-name><surname>Kenski</surname>, <given-names>K.</given-names></string-name>, <string-name><surname>Coe</surname>, <given-names>K.</given-names></string-name>, &amp; <string-name><surname>Harwood</surname>, <given-names>J.</given-names></string-name> (<year>2017</year>). <article-title>Incivility and political identity on the Internet: Intergroup factors as predictors of incivility in discussions of news online</article-title>. <source>Journal of Computer-Mediated Communication</source>, <volume>22</volume>(<issue>4</issue>), <fpage>163</fpage>&#8211;<lpage>178</lpage>. DOI: <pub-id pub-id-type="doi">10.1111/jcc4.12191</pub-id></mixed-citation></ref>
<ref id="B38"><label>38</label><mixed-citation publication-type="journal"><string-name><surname>Rossini</surname>, <given-names>P.</given-names></string-name> (<year>2020</year>). <article-title>Beyond incivility: Understanding patterns of uncivil and intolerant discourse in online political talk</article-title>. <source>Communication Research</source>, <elocation-id>0093650220921314</elocation-id>. DOI: <pub-id pub-id-type="doi">10.1177/0093650220921314</pub-id></mixed-citation></ref>
<ref id="B39"><label>39</label><mixed-citation publication-type="journal"><string-name><surname>Rossini</surname>, <given-names>P.</given-names></string-name>, &amp; <string-name><surname>Maia</surname>, <given-names>R. C. M.</given-names></string-name> (<year>2021</year>). <article-title>Characterizing Disagreement in Online Political Talk: Examining Incivility and Opinion Expression on News Websites and Facebook in Brazil</article-title>. <source>Journal of Deliberative Democracy</source>, <volume>17</volume>(<issue>1</issue>), <fpage>pp</fpage>. <fpage>90</fpage>&#8211;<lpage>104</lpage>. DOI: <pub-id pub-id-type="doi">10.16997/10.16997/jdd.967</pub-id></mixed-citation></ref>
<ref id="B40"><label>40</label><mixed-citation publication-type="webpage"><collab>Save the Storks</collab>. (<year>2019</year>). <source>The difference between being pro-life and anti-abortion</source>. <uri>https://savethestorks.com/2019/03/the-difference-between-being-pro-life-and-anti-abortion/</uri></mixed-citation></ref>
<ref id="B41"><label>41</label><mixed-citation publication-type="confproc"><string-name><surname>Sharma</surname>, <given-names>E.</given-names></string-name>, <string-name><surname>Saha</surname>, <given-names>K.</given-names></string-name>, <string-name><surname>Ernala</surname>, <given-names>S. K.</given-names></string-name>, <string-name><surname>Ghoshal</surname>, <given-names>S.</given-names></string-name>, &amp; De <string-name><surname>Choudhury</surname>, <given-names>M.</given-names></string-name> (<year>2017</year>). <article-title>Analyzing ideological discourse on social media: A case study of the abortion debate</article-title>. In <conf-name>Proceedings of the 2017 international conference of the computational social science society of the Americas</conf-name>, <fpage>1</fpage>&#8211;<lpage>8</lpage>. DOI: <pub-id pub-id-type="doi">10.1145/3145574.3145577</pub-id></mixed-citation></ref>
<ref id="B42"><label>42</label><mixed-citation publication-type="book"><string-name><surname>Silge</surname>, <given-names>J.</given-names></string-name>, &amp; <string-name><surname>Robinson</surname>, <given-names>D.</given-names></string-name> (<year>2017</year>). <source>Text mining with r: A tidy approach</source>. <publisher-name>O&#8217;Reilly Media, Inc</publisher-name>. DOI: <pub-id pub-id-type="doi">10.16997/jdd.50</pub-id></mixed-citation></ref>
<ref id="B43"><label>43</label><mixed-citation publication-type="journal"><string-name><surname>Stromer-Galley</surname>, <given-names>J.</given-names></string-name> (<year>2007</year>). <article-title>Measuring deliberation&#8217;s content: A coding scheme</article-title>. <source>Journal of Deliberative Democracy</source>, <volume>3</volume>(<issue>1</issue>).</mixed-citation></ref>
<ref id="B44"><label>44</label><mixed-citation publication-type="journal"><string-name><surname>Suiter</surname>, <given-names>J.</given-names></string-name> (<year>2018</year>). <article-title>Deliberation in action&#8211;Ireland&#8217;s abortion referendum</article-title>. <source>Political Insight</source>, <volume>9</volume>(<issue>3</issue>), <fpage>30</fpage>&#8211;<lpage>32</lpage>. DOI: <pub-id pub-id-type="doi">10.1177/2041905818796576</pub-id></mixed-citation></ref>
<ref id="B45"><label>45</label><mixed-citation publication-type="journal"><string-name><surname>Sykora</surname>, <given-names>M.</given-names></string-name>, <string-name><surname>Elayan</surname>, <given-names>S.</given-names></string-name>, &amp; <string-name><surname>Jackson</surname>, <given-names>T. W.</given-names></string-name> (<year>2020</year>). <article-title>A qualitative analysis of sarcasm, irony and related# hashtags on twitter</article-title>. <source>Big Data &amp; Society</source>, <volume>7</volume>(<issue>2</issue>), <fpage>2053951720972735</fpage>. DOI: <pub-id pub-id-type="doi">10.1177/2053951720972735</pub-id></mixed-citation></ref>
<ref id="B46"><label>46</label><mixed-citation publication-type="journal"><string-name><surname>Theocharis</surname>, <given-names>Y.</given-names></string-name>, <string-name><surname>Barbera</surname>, <given-names>P.</given-names></string-name>, <string-name><surname>Fazekas</surname>, <given-names>Z.</given-names></string-name>, &amp; <string-name><surname>Popa</surname>, <given-names>S. A.</given-names></string-name> (<year>2020</year>). <article-title>The Dynamics of Political Incivility on Twitter</article-title>. <source>SAGE Open</source>, <volume>10</volume>(<issue>2</issue>), <fpage>1</fpage>&#8211;<lpage>15</lpage>. DOI: <pub-id pub-id-type="doi">10.1177/2158244020919447</pub-id></mixed-citation></ref>
<ref id="B47"><label>47</label><mixed-citation publication-type="journal"><string-name><surname>Wang</surname>, <given-names>M. Y.</given-names></string-name>, &amp; <string-name><surname>Silva</surname>, <given-names>D. E.</given-names></string-name> (<year>2018</year>). <article-title>A slap or a jab: An experiment on viewing uncivil political discussions on Facebook</article-title>. <source>Computers in Human Behavior</source>, <volume>81</volume>, <fpage>73</fpage>&#8211;<lpage>83</lpage>. DOI: <pub-id pub-id-type="doi">10.1016/j.chb.2017.11.041</pub-id></mixed-citation></ref>
<ref id="B48"><label>48</label><mixed-citation publication-type="book"><string-name><surname>Wright</surname>, <given-names>S.</given-names></string-name>, <string-name><surname>Graham</surname>, <given-names>T.</given-names></string-name>, &amp; <string-name><surname>Jackson</surname>, <given-names>D.</given-names></string-name> (<year>2017</year>). <chapter-title>Third space and everyday online political talk: Deliberation, polarisation, avoidance</chapter-title>. In: <source>The 67th Annual Conference of the International Communication Association</source>, <day>25&#8211;29</day> <month>May</month> 2017, <publisher-loc>San Diego, USA</publisher-loc>.</mixed-citation></ref>
<ref id="B49"><label>49</label><mixed-citation publication-type="webpage"><collab>YouGov</collab>. (<year>2022</year>). <source>Pro-life, anti-abortion, or anti-choice: what label Americans use for people who oppose abortion</source>. <uri>https://today.yougov.com/topics/politics/articles-reports/2022/06/17/pro-life-anti-abortion-or-anti-choice-what-label-a?amp;amp=</uri></mixed-citation></ref>
<ref id="B50"><label>50</label><mixed-citation publication-type="book"><string-name><surname>Ziegler</surname>, <given-names>M.</given-names></string-name> (<year>2023</year>). <source>Roe: The History of a National Obsession</source>. <publisher-name>Yale University Press</publisher-name>. DOI: <pub-id pub-id-type="doi">10.12987/9780300271874</pub-id></mixed-citation></ref>
</ref-list>
</back>
</article>