Judging Deliberation: An Assessment of the Crowdsourced Icelandic Constitutional Project

This study explores deliberation as a lived experience between individuals engaged in putatively deliberative practices. While face-to-face deliberation is well documented, there are fewer empirical studies that address its online counterpart. The authors review current theoretical conceptualizations and operationalize a measure of deliberation, and then apply the measure to the case of the debate fostered by the Constitutional Council online public platform dedicated to drafting the Icelandic constitution—the first ‘crowdsourced’ constitutional project in the world. This is the first effort to both quantitatively and qualitatively evaluate the nature of deliberation in the case of Iceland. Generally, this exploration is meant to identify and analyze markers of deliberation in a setting that aspires to foster such exchanges. The paper concludes by discussing the implications of this work for future political theory and related empirical investigation.

Civil society is fundamentally based on the interchange of ideas and the willingness to consult and cooperate. As Tocqueville notes, civil society is a central feature of democratic life (Tocqueville 1964) because it provides citizens a way to filter and refine their ideas, as well as mobilize others for the purpose of influencing political outcomes. In participatory democracy, a strong civil society affords an array of guiding voices and policy options for the state and empowers shared interests (Fishkin 2018). An exchange of ideas is thus fundamental to the dialectic of interest formation and mobilization. On the other hand, the public nature of civil society outcomes requires a set of preconditions that would theoretically feed into, if not directly result in a just and equitable policy outcome. Ideally, informed viewpoints should be exchanged with an eye to the public good, in the vein of what Tocqueville called 'self-interest rightly understood.' But not every exchange is deliberative, nor does every deliberative exchange have a clear outcome, let alone a just outcome. It is thus possible to theoretically differentiate between a process that is deliberative, a process that is deliberative and decisive (results in a solution to a problem), and a deliberative process that is deliberative, decisive, and just. There is reason to believe that deliberative proceedings which meet a set of conditions have a much higher chance of resulting in a desirable outcome that is essentially connected to the common good. In the following paper, we empirically investigate online discussion to identify the presence or absence of exchange, a fundamental element of effective deliberation, where effectiveness is the capacity to deliver informed judgment with an eye to the common good.
We address theoretical and empirical questions about deliberation by first identifying the parameters of normative deliberation and then empirically applying this model to an existing, online case. The online setting is important because, as Albrecht argues, ' of the three major political practices-information provision, deliberation, and voting-deliberation can be seen as the most challenging issue in analyzing the Internet's impact on politics ' (2006: 63). Maboudi and Nadi (2016) provide an important empirical analysis of online deliberation, arguing convincingly that online deliberation matters for policy outcomes. They show that ' online public feedback significantly affected the probability of constitutional changes' and that, when constitution drafters agree on the design of the constitution, 'popular preferences would be reflected in the constitution' when ' citizens agree on the most salient issues.' While Maboudi and Nadi's findings are persuasive, they point to the potential that Egypt's experience may not be reflective of other nations, suggesting a need for further case studies. Muradova's (2021) analysis of the Irish Citizen's Assembly contributes just such a case study, arguing that deliberation can effectively lead participants to take other perspectives and result in reflective political reasoning. While not a study of online deliberation, Murdova's work nonetheless contributes to the growing literature about online deliberation by offering theoretically sound empirical measures to evaluate effective deliberation.
We build on this research by exploring online deliberation in different national contexts with empirical measures of deliberation meant to capture the give and take represented by Murdova's concept of 'perspective taking,' which she defines as ' actively imagining others ' experiences, perspectives, and feelings' (2020:4). The case we selected is the online debate fostered by the Constitutional Council online public platform (CCPP) 1 dedicated to drafting the Icelandic constitution, the first ' crowdsourced' constitutional project in the world (Morris 2012). The internet set-up was intentionally intended to involve direct public participation. It was moderated and non-anonymous. The online portion was part of a multi-faceted process involving several steps of public participation (in the creation of the committee, for instance). In that sense, the Icelandic model was intended to include broad participation and involved a variety of deliberative methods which we will briefly discuss below. Previous research on this case measured the degree to which the online debate resulted in changes in the constitutional draft (Hudson 2018), as well as inclusivity of the process (Landemore 2015) and the success of group reasoning (Landemore 2017). These valuable contributions leave unanswered a fundamental question: what was the quality of the deliberation that led to the online proposals that were (or could have been) incorporated into the constitutional text? If the public process is flawed, then the inclusion of proposals resulting from this process would not be a beneficial outcome for democratic life. Thus, we explore the quality of deliberation on the Icelandic Constitutional Council online platform and pay attention to the kind of interaction happening within and between posts in threads related to the draft.

Analyzing Deliberation
The increasingly sophisticated research agenda of democratic theory has progressively combined the procedural and the developmental aspects of democratic civic life (Chambers 2009;Held 2006). Deliberative theory is, in many respects, the theoretical ground for this combination and for the effort to develop mechanisms to both institutionally structure the reasoned exchange of policy solutions and the distribution of public goods, as well as to increase the capacity of the citizenry to participate in informed and persuasive public exchanges (see Cohen 1989;Elster 2012;Fishkin 2019;Landemore 2013;Marti 2006;Rawls 1993). Well-structured deliberation promises to increase the value of democracy as a public good by constituting public involvement, exchanges, and outcomes around 'the unforced force of the better argument' (Habermas 1996). A significant amount of recent research empirically tests the theoretical arguments about the transformative qualities of good deliberation (Ackerman & Fishkin 2004;Davis 2005;Ekenberg 2017;Fishkin & Luskin 2005;Gibson et al. 2003;Mendelberg 2002). Much of this research has focused on different aspects of deliberation, and it has selectively used conceptual markers like consensus (Dryzek 2000;Niemeyer & Dryzek 2007), beliefs (Gastil & Dillard 1999a;1999b), respect (Gutman & Thompson 1996, or outcome as the key determinants of deliberation. Given this conceptual variety, it is perhaps not surprising that empirical results have been inconsistent (Thompson 2008), and evidence has supported both an improvement and a decay of the quality of deliberative exchanges, depending on group dynamics (Isenberg 1986;Kerr & Tindale 2004) or degree of political engagement in relation to group discussion (Gastil, Deess & Weiser 2008;Mutz 2006). The basic puzzle of theoretical and empirical inquiry remains the same: under what conditions can we develop collective intelligence that serves the common good for the purposes of justice and fairness (Landemore 2012;Marti 2006;Talisse 2009)? This question is complicated by the rise of new methods of interactive exchange, specifically online platforms. More recent discussions of the virtues of democracy have refocused on the possibility of enhancing the democratic sphere by folding in new voices with the help of widely available internet tools (Coleman & Blumler 2009;Dahlgren 2005;Wright 2012), and hopeful voices argued for the rise of e-democracy (Noveck 2004). The online deliberation debate has generally pitted cyber-optimists, who argued that the internet would revolutionize deliberative possibilities and ' democratize technology' to create virtual public platform (Naisbitt 1991;Rheingold 1993) against cyber-realists (see Shane, 2004), who generally agreed with Margolis & Resnik (2000;see also Davis 2005;Gibson et al. 2003) that potential revolutionary effects would be mitigated by the persistence of many familiar and real problems: 'The utopian vision of a worldwide agora that would revitalize democracy has to confront the harsh reality of lawsuits and regulations, commerce and entertainment, political parties, organized interest groups, political activists, and, most important, masses of bored and indifferent citizens' (Margolis & Resnik 2000: 14, 22). In response to this debate, researchers such as Wright (2012) have argued that this so-called revolution/normalization frame has funneled research questions and normative perspectives to the detriment of a more nuanced assessment of when, where, and how online discussion actually happens. Wright (2012) further argues that this conceptual schism has possibly generated an unduly pessimistic outlook on the quality and prevalence of online political debate, and he invites a more nuanced approach in future research. We pursue the present analysis with an awareness that the episode we capture and analyze is part of a larger process and may be a step in an incremental and multifaceted process that broadens deliberative content. The subject of this analysis is the CCPP of the crowdsourced initiative of drafting a new Icelandic constitution, a case that presents us with the opportunity to analyze an important new approach to constitutional drafting, that also benefits from broad public participation, public information sessions preceding the online deliberative portion, and online moderation in a specifically designed forum.

Why the Icelandic constitutional draft?
In response to the 2009 European banking crisis and significant economic losses, Iceland responded in a unique way: it decided to reevaluate the underlying political principles of the Icelandic state (Gylaffson 2011;Oddsdottir 2014;Olafsson 2016). The purpose of this reassessment was to inquire into the nature of the relationship between political principles and economic outcomes. In sum, Icelanders wanted to know how stagnant politics gave rise to rampant capitalism, the kind that undermines the democratic principles of a state. In late 2008 and early 2009, thousands of Icelanders took to the streets in what amounted to a social movement dubbed the 'Pots and Pans Revolution,' and private citizens were mobilized into a grassroots organization called 'the Anthill,' which organized the random selection of 1,500 citizens who were asked to participate in an open discussion day and define core Icelandic values. Its representatives addressed the Icelandic Parliament (Althingi) and pressed for a Constitutional Convention, arguing among other things, that Iceland's constitution still harkened back to the constitution Iceland had when it was still a client of the Danish state (Olaffson 2016). As Jon Olafsson noted, 'the current constitution is closely related to the former Danish constitution and indeed was never meant to be more than a provisional constitution, created in a hurry as Icelanders founded their own republic during the German occupation of Denmark in World War II' (Olafsson 2018; see also Bergsson & Blokker 2014: 155). The public call was for a serious and radically democratic project to rewrite the constitution, since 'the new Icelandic republic never autonomously wrote a constitution to match the independent state' (Bergsson & Blokker 2014: 155).
In 2009, the post-crash government led by Prime Minister, Johanna Sigurdardottir (Social Democratic Alliance) agreed to start the redrafting process ('Act on a Constitutional Assembly no. 90/2010,' and appointed a Constitutional Committee of seven people charged with organizing a National Forum. 2 As a result, 950 citizens were selected at random from the national registry (Gylfason 2013) and tasked with identifying the basic values that should be reflected in the new constitution (see Hudson 2018). After a day of deliberations in November 2010, the National Forum decided that a new constitution was necessary and should reflect key values such as 'honesty, justice, equality (egality) and freedom, and then love, responsibility, family and democracy.' The meeting also suggested some criteria for the process that followed (Helgadottir 2014). 3 In October 2010, the government also held a national election for a consultative Constitutional Assembly 'to which 25 individuals were elected from a roster of 522 candidates from all walks of life, most of them with no particular political or special interest affiliations' (Gylfason 2013; see also Burgess & Keating 2013). All voting age Icelanders were allowed to both vote and stand for election with the exception of the President, ministers, and members of Parliament who could vote but could not stand for election on this body. 4 Generally, political parties were excluded from the drafting process, although the new constitution would need to be approved by two successive Althingi votes, with a general election in-between.
The Constitutional Assembly was meant to draft the actual text of the new constitution, starting with the values underscored by the National Forum, but technical details of the election process were challenged in the Icelandic Supreme Court, which declared the results null and void in January 2011. 5 Moving the process forward, the Icelandic Parliament created a Constitutional Council instead, by appointing those same 25 individuals who had received the most votes, and charged them with creating a draft in four months (6 April-29 July 2011). Although there were objections to this process, the Constitutional Council pursued its work (Gylfason 2011;. The Council was also required to actively elicit public input (Thorarensen 2014), and it used popular social media such as YouTube, Twitter, Facebook, and Flickr to post videos, stream proceedings, upload information, and updates of their work and solicit comments.
The most relevant site for input was maintain by the Council at http://www.stjornlagarad.is/erindi/. The council invited submissions, in the form of brief opinion pieces, on topics relevant to the constitutional debate and it received 350 submissions from 218 individuals and associations (Gylfason 2011;Helgadottir 2014). The public was then able to respond to these opinion pieces through a Facebook plugin that appeared below each opinion piece. To participate, posters had to provide their actual names and addresses; their login names were their actual names. As Helgadottir points out, participation reflected more men than women, and more individuals than interest groups: 13 per cent of submissions came from women, 77 per cent from men, and 10 per cent from civic organizations (2014). Because the site was maintained and moderated by the Constitutional Council, Council members responded directly to the comments in the Facebook plugin threads.
The constitutional draft was presented to the Icelandic Parliament on July 29, 2011. Fifteen months later, after further reviews by Icelandic lawyers and parliamentary filibusters (Gylfason 2013), the draft was submitted to public vote on October 20, 2012, and the results generally endorsed the new constitution. Voter turnout was 49 per cent, and 67 per cent of the electorate voted in favor of the bill and the many supported key individual provisions such as national ownership of natural resources (83 per cent said 'Yes') and equal voting rights (67 per cent said 'Yes') (Gylfason 2013). The referendum results sparked some controversy, especially regarding the phrasing of some key questions, 6 and how the respective answers should be interpreted, and regarding the extent to which the results should be binding on the Althingi given that the referendum was framed as a ' consultative' step (Gylfason 2013;Thorarensen 2014).
The outcome of the referendum did not quell the opposition, and three years after the drafting process started, the Constitutional draft was tabled by the newly elected Althingi members (Thorarensen 2014). Those opposing the new constitution, representing the Independence and Progressive parties tended to reflect a position stated by Birgir Armansson, MP with the Independence Party: 'The idea of sidelining parliament by electing a special constitutional council, we were always against that. (…) From our point of view, if you believe you are in a democratic society the way to change the constitution is according to the constitution, not trying to sideline the normal procedures' (quoted in Calatayud 2019).
The delay resulted in a relatively strong public reaction that kept the issue alive (Gylfason 2013;, and by late 2017, 55% of Icelanders considered getting a new constitution very important or rather important, according to a poll reported then in the Icelandic press. On this background, the effort to compel the Icelandic Parliament to ratify the referendum remains a significant political and social movement in Iceland, and the Althingi periodically revives the constitutional reform effort (Hudson 2018).
From our analytical perspective, the uniqueness of the process to find the Constitutional Council drafters was twofold: first, the drafters were selected through a broad public appeal, and the eventual drafting council was made up of twenty-five people with a variety of backgrounds (artists, lawyers, farmers), and second, as the council was drafting each article of the constitution, an online Facebook plugin was opened up and maintained (by the Constitutional Council) in order to receive public input to virtually each word in the text of the constitution's draft. The drafting committee meeting would start each day with a discussion of the comments received the previous day. The person in charge of managing this site was Finnur Magnusson, and this online feedback process lasted a few months. As members of the Constitutional Council noted themselves, the Facebook plugin was the main interface on which the council received substantial comments (Sigmundsdottir 2011). 7 Our analysis retraces the steps of the deliberative process that was part and parcel of the drafting of the new Icelandic constitution. The Constitutional Council's task was to produce a draft in only a few months, and the council's intention was to consult with the public as much as possible over that time. To that end, the Council live-streamed their discussion online, posted minutes, and set up the Constitutional Council Public Platform (CCPP, operated by a Facebook plugin) and prompted public comments by allowing brief commentary pieces/ proposals on various topics related to each article in the draft. Gylfason (2011) reports that the Council received 323 proposals and a total of 3,600 comments through various sources. This paper uses all 311 proposals currently available and the 1,479 comments posted in response. Helgadottir reports that ' of the submissions on the webpage, 13% come from women, 77% from men and 10% from organizations. While the age of those participating is not readily available, 40 submitters were chosen at random and their age examined. Out of that sample, very few were young people: Around 80% were between 40 and 65 years of age. Finally, a few foreign citizens made suggestions via the web-page' (Helgadottir 2014).
Online, public deliberation was nested in a process involving multiple steps that have been described by both members of the Constitutional Council (Bergmann 2016;Gylfason 2016;Oddsdottir 2014), and analyzed by academics (see Landemore 2015;Olafsson 2016;Suteu 2015;Valtysson 2014). The context of the drafting process is significant: Iceland is a small country with a homogeneous population, high levels of education, and very high internet access (reported at 96 per cent by Kelly et al. 2013). Iceland has also very high voting rates (88 per cent average since 1964; Kelly et al. 2013).
As a case study for deliberation, the Icelandic constitution draft presents several advantages: the process is set in a small, homogeneous country with high levels of participation and high voter turnout. Moreover, as described above, the process involved several steps of public consultation that follow, at least to some extent, several methods of deliberative consultation described and categorized by Fishkin in his recent work, Democracy When the People Are Thinking (2018). For instance, the creation of the Constitutional Council was preceded by the National Forums, which were events to which randomly selected citizens were invited to sit together (usually at round tables in large public spaces) and discuss their opinion and reasoning toward some consensus regarding important issues or values that were then rank ordered by frequency. These Forums looked much like what Fishkin and Ackerman call 'Deliberation Day' (Fishkin 2019: 175)). Additionally, the Constitutional Council was formed by both inviting particular individuals and random selection from the voter registry along the lines of a citizen jury with a random polling element, both methods identified by Fishkin as deliberative elements. Furthermore, the online discussion element is type of 'self-selected listener opinion poll' (SLOP), a term coined by Norman Bradburn, former Director of the National Opinion Research Center (Gawiser & Witt 1994). Participants to SLOPs are those individuals interested enough in a topic to choose to participate by registering their opinions online (or in other forms). In aggregate, these consultation procedures were meant to increase the degree of openness, transparency, and participation, which the Constitutional council held to be the three key values of the process. Each has shortcomings, as Fishkin points out (2018), but their cumulative effect is an important context to this study, since it might affect the degree to which the online portion was more or less deliberative. Keeping in mind that online deliberation was one piece of the process, the deliberative puzzle is not complete without a qualitative assessment of the argumentative patterns within the online public input.

Measurement and Methods
Our motivating question is whether the CCPP exhibited traits of normative deliberation. Mercier and Landemore (2012) outline the characteristics of good deliberative performance and draw on the Argumentative Theory of Reasoning to identify contexts when reasoning will work or fail. What does a good deliberative performance look like? An activity is deliberative to the extent that reasoning is used to gather and evaluate arguments for and against a given proposition (Mercier & Landemore 2012). The content of statements is less important than the cognitive activity of reasoning used to produce them. Next, there must be an exchange between parties, or what Mercier and Landemore term a 'feedback loop,' for deliberation to occur. In addition to exchanging statements, parties in deliberation must evaluate opposing claims as they are understood-the internal phase of deliberation. Ideally, the performance of deliberation is an interplay of internal and external reasoning, in which interlocutors both think of and offer to others an assessment of statements, supported by evidence. Here, we operationalize these components of deliberation to determine the extent to which the content of the CCPP matches the normative model of deliberation. We implement measures used in previous work (Loveland & Popescu 2011) to determine the degree of deliberation occurring on the CCPP.

Data and Coding
The data for this study come from a period of three months, April-June 2011, on the Public Platform hosted by the Constitutional Council (CCCP; Stjornlagarad.is). In the three months, the website produced 311 submissions that invited comments. With the help of website administrators with deep access, we collected all 311 online CCPP threads from the website of the Constitutional Council. The online feedback set-up followed a pattern: an initial article/opinion piece was posted on the website, and the public was invited to comment on that piece in relation to what the constitution draft should contain. As public opinion was intended to feed into the language of the draft, the initial posts were not drafts of the constitutional articles, but rather commentary pieces related to issues under review in the Constitutional Council. Consequently, the discussion was intended to help articulate the articles in the draft. The opinion pieces were written by various interested parties like academics, civil society leaders, or Ngo representatives. As an example, the initial post for a discussion on constitutional housing rights was written by the representative of an Icelandic fair housing NGO advocating for housing as a basic human right and the end of homelessness through subsidized housing. The comments were supposed to respond to the arguments in this page-long article.
Two sets of files were created: one in the original Icelandic and the other translated into English with Google translate, in order to facilitate basic topic recognition. Out of the 311 threads we identified and chose to analyze the three longest, which seem to hold the promise of most deliberative qualities. These three files were carefully translated into English by a native Icelandic speaker. 8 In the three months the online platform was available, the 311 threads produced 1,479 individual posts. We are aware of the possibility that some posts are missing, as there is a discrepancy between the data we found and the count reported by Hudson, who puts the number of posts at 1,575 (2018). While the source of the discrepancy is unclear, the difference in data is not significant enough to substantially affect the conclusion. As previously reported by Hudson (2018), the 311 introductory posts/ commentary pieces came from 204 individuals, so some individuals provided more than one opinion article that was then open for public discussion.
Individual names were also counted in each thread only once in order to obtain the number of individual posters for each of the 311 threads. The number of posters was based on individual names because the website did not allow anonymous posters. It is conceivable that posters misreported their names, which would not affect the data, or that they entered comments under more than one name, which would affect the data. In order to report the total number of individual posters across all 311 threads, we also relied on previously existing data collected for the same threads by Hudson (2018).
The threads were fairly heavily moderated-the moderator would intercede to point out that discussion should be civil and answers kept short, as well as remove posts considered offensive. The threads had to be managed-for instance, posts that were too long needed to be sometimes forced on the website through IT/ moderator intervention which in some cases produced a substantial amount of lag time in posting.
The coding scheme we applied relies on a previous analysis of a newspaper website for a mid-size northeastern city in the US (Loveland & Popescu 2011) and complements the Argumentative Theory of Reasoning (Mercier & Landemore 2012), as well as Muradova's perspective taking theory (2021). The eleven variables we developed are intended to measure the deliberative character of each of the posts, as well as the resulting threads. According to previous research and theory, deliberation must include at least two actors who address a common topic, and the posts must contain argumentation (Chambers 2004;Delli Carpini et al. 2004). Consequently, posts were coded as seeds and responses. Following concepts developed by Fishkin (1995) and Morrell (2005), a post is a seed if another post in the same thread responded to it, and a post is a response if it makes reference to a topic initiated in a prior post. Posts that encouraged a response by directly seeking information were coded as 'seeking. ' We theorized that posts seeking information might be more likely to elicit a response, and therefore have the potential to be seeds in the deliberative exchange.
Posts were also coded for the presence of argumentation, which was identified when posts made claims and presented conditions of validity for those claims with the intent to rationally persuade readers (Benhabib 1994;Gastil 2008). Here, our work follows that of Mercier and Landemore (2012), who study the success of reasoning at the collective level: what we refer to as the 'thread level. ' We also coded whether a post was in agreement with, or contradictory to, a previous post. We also included a measure regrading whether or not a post provided information. We theorize that bringing in information aids the deliberative process and is likely to spur or enhance argumentation in following posts.
Beyond conditions for argumentations, we also looked at the emotive aspects of deliberation. Previous research (Rosenberg 2007) emphasizes that respect sets the stage for effective deliberation. Importantly, other studies (Ferber et al. 2006) point out that online deliberation is not particularly apt to foster a courteous exchange, and that 'flaming' is a common occurrence. In order to lend some nuance to the qualitative analysis, we went beyond the flaming-polite duality to register the 'valence' of the post: the negative, neutral, or positive overall tone. We coded posts as positive when they were encouraging, praising, or optimistic. For example: 'Trying to help people keep their houses -that is a good idea' (thread #303). Neutral posts are those that were not obviously positive or negative. Examples of a neutral posts include those that invite posters to read an article, describe laws that apply in a case, or the process by which something happened. Negative posts berated an idea or were overtly snide or critical. For instance: 'This is a terrible idea, Illugi. I just can't believe it will pass. First, it is ridiculous to wait five years. Why on Earth should we wait five years for this change but not others? …If this will be the outcome, I cannot take any proposals from the Constitutional Council seriously. I know the same applies to others. I'm sorry, but you simply have to deal with it' (Thread #100).
Some posts were generally oriented to the discussion, while others targeted people. In order to capture important aspect of targeting, both negative and positive, we differentiated between personal attack and personal praise. If a post was negative and focused on another poster, we coded it as a personal attack. For example, in thread #208, we deemed this post to meet the criteria of personal attack: 'Solveig, are you proud of being a spokesperson for discrimination and inequality in Iceland?' If a post was positive and directly engaged another poster, we coded it as personal praise.
The eleven variables we developed (seed, response, valence, argument, seeking, information, personal attack, personal praise, inflammatory, contradiction, agreement) allowed us to analyze post-level data as well as threadlevel data. At the post level, the relevant questions are whether or not a post seeks information or provides argumentation, if is it in agreement with another post, and what is its valence. There is also the possibility of thread level analysis based on aggregate measures and focused on the interactional elements of the variable counts. Our findings, presented below, are descriptive and meant to provide context for determining the degree to which the forums were deliberative.

Findings
The analysis is exploratory and descriptive, based on a non-random sample of the threads, and meant to fill a gap in previous research about the Icelandic constitution drafting process. While Hudson (2018) looked at general patterns of interaction, we examine post-and thread-level characteristics. Our primary question is to determine if active online forums can be reasonably called deliberative according to the characteristics outlined above. Therefore, we do not test hypotheses about the correlates of deliberation, but rather report on what we found in our observation of three exceptionally long threads that attracted a high degree of participation. We intentionally choose what appears to be the threads that generated the most interest (the longest threads with multiple participants), and we analyze the interaction to see if we do find instances of deliberation as it is understood in political theory.

Post-Level Descriptive Statistics
We first describe the average post for the three threads combined. Because of the high number of posts per thread, the three threads we analyzed amount to a sizable 15 per cent of the total number posts across all 311 threads. Of the 229 posts that we coded across the three threads, about 9 per cent started a conversation, and an impressive 97 per cent responded to seed posts, while 27 per cent were seeking information ( Table 2). As in a previous analysis of an online forum, based on these statistics and our qualitative observations, we discern a pattern: the first post in the thread generates multiple individual responses, but the interaction rarely becomes more complex. Previous research conclusions hold in this case as well, as 'typically, there are not multiple contributors making a claim, hearing a response, and then reacting to those responses. Instead, many contributors make a post and then leave the conversation' (Loveland & Popescu 2011).
Regarding the content of the posts across the three threads, in terms of valence, the mean was 2.35 out of 3 (where a score of 3 is negative valence, 2 neutral and 1 positive) suggesting that the tone tended to be slightly negative, and few responses were coded as positive. Eighteen per cent of the posts were personal attacks, and less than 3 per cent offered personal praise. Only 15% per cent of the posts expressed agreement with another post, while 49 per cent expressed disagreement. While high levels of disagreements do not necessarily imply low levels of deliberation, disagreement far outpaced agreement.

Thread-Level Analysis
The three threads we selected contain the largest number of posts and participants, representing 15 per cent of the total posts across the 311 threads. We counted the number of participants individually, by name, without double counting the same name, in order to differentiate between the number of posts and the number of participants. The threads we analyze are unique relative to the overall forum. Table 1 presents basic descriptive statistics. The average number of participants per thread was 3.35, which is rather low. Out of the 311 threads, 49 had 0 posts beyond the prompt, 44 consisted of 1 post, and 52 were 2 posts long. In sum, 145 threads out of the 311 have 2 posts or less (47%). Out of the 311 threads, 287 had less than 10 posts (92%). The average number of posts per thread was 4.59, and 101 threads had more than the average number of posts (32%). In this context, the three threads we selected (#100, #208, # 303) were well beyond average on both comments and participants: thread #100 had 83 comments and 23 participants; #208 had 98 comments and 63 participants; and #303 had 51 comments and 23 participants. By comparison, the fourth most popular thread (#157) had only 25 comments and 5 participants. Our sample of the longest threads is, indeed, not representative of all of the threads, but this selective sampling allows us to determine if the most active threads were deliberative.
It is no surprise that some topics generate far more interest than others, and what is of concern here is whether the most active threads can be considered deliberative. We return to questions about what the overall descriptive measures can tell us about the deliberative nature of the entire set of forums in the discussion. Measures for the three threads we analyzed in detail are presented in Table 2.
The three threads differed only slightly in either thread-or post-level measures. In each thread, most posts were responses, and disagreement was more common than agreement. Threads #100 and #208 were both related to a discussion regarding the national church, the Christian profile of Iceland, and the free exercise of religion, while thread #303 covered the topic of housing as a constitutional right. While the first two threads included much discussion based on personal opinion, thread #303 invited a number of legal questions related to property rights, banks, loans, lending practices and financial institutions-all topics related to more technical knowledge and hence invited more technical questions.
Interestingly, thread #303 exhibited much higher levels of information seeking, which might have to do with the nature of the discussion which was related to housing rights (including financial assistance, taxation, mortgages, banking, etc.), which is a more technical matter. The article starting the conversation was authored by Andrea Olafsdottir, whose signature revealed that she was the Chair of the Board for the Household Confederation, an organization in support of housing rights. Our qualitative observation supports the fact that this discussion was infused with more legal and bankingrelated details. While more information was sought, no more information was provided than the average in the other threads (around 26%).
Examining the emotive content of threads, we found a marked difference in percentage of inflammatory posts. The most inflammatory thread is #208, a thread dedicated to a discussion of whether or not Iceland is a Christian nation, the extent to which that should be reflected in the constitution, and whether or not the inclusion of a state church requires a national referendum. Interestingly, a nuance not captured by our coding system was the degree to which posters responded sarcastically. The qualitative analysis revealed a high degree of sarcasm on multiple posts, like those that suggested including in the constitution the fact that most Icelanders are blond, or white, or a particular gender or sexual orientation, all meant to be sarcastic reductio ad absurdum comments. An example of this type of comments is: 'yes, we should include: Icelanders are white, Christians, and geniuses in international trade and money matters!' Conversely, the rather technical and far less controversial discussion of housing rights registers only 16 per cent inflammatory comments, comparatively less than the 38 per cent for #208 or the 24 per cent for #100. Although we analyzed only three threads, we also looked into exploratory multivariate analysis. Among other factors, deliberation requires multiple parties to participate, to be invested in the interaction, and to be on topic. According to Beauchamp, a key element is 'mutual consideration of conceptually related ideas' (Beauchamp 2020: 1). To account for these requirements, we developed a measure that is straightforwardly applicable to online forums. It is quite common for threads to be composed of single posts by multiple posters, meaning that no one participant posted more than once in the thread. It is also quite common for posts to be ' off-topic,' meaning they are not clearly a response to a prior post in the thread. Ideally, a deliberative thread will have multiple posters, who post more than once, and post on topic. Our measure is based on this ideal and is used to compare threads according to these characteristics.
For each thread we calculated the rate of posts per posters, and all had more posts than posters, which suggested that some posters made multiple posts. As we have argued in previous research, 'simply returning does not indicate investment in the conversation, and to account for this we rely on our measure of "response" outlined above. The greater the proportion of posts which are "on topic," the more comfortable we are calling the interaction deliberative' (Loveland & Popescu 2011). We acknowledge that our measure captures just one aspect of a multidimensional concept. Nonetheless, others have identified the value of analyzing individual components of deliberation even if no measure can capture it ' as a whole or all in one go' (Bachtiger & Parkinson 2019: 151), and Beauchamp writes that 'mutual consideration of conceptually related ideas' is the 'the core process of deliberation' (2020: 1). Hence, our measure of threadlevel deliberation is calculated as follows: Where D indicates the deliberative quality of the thread, P is the number of posts, C is the number of unique contributors to the thread, and R is the proportion of thread posts which responded to a prior post.
We calculated this deliberation score for each thread, and we found that thread #100 had a score of 3.39, #208 scored 1.49, and #303 measured at 2.09. The conclusion is that among this limited set of three threads, #100 is the most deliberative, while #208 is the least deliberative. Qualitatively, this aligns with the observation that the issue of whether or not the constitution should reflect that Iceland is a Christian nation appears to be the most contentious issue, which invited the most inflammatory comments, and a good number of sarcastic comments, which complicated the coding scheme. Of course, this is just one aspect of deliberation, and the scores must be interpreted relative to other characteristics of the threads. While there is a fair amount of give and take in each thread, we cannot ignore the relatively large number of posts that were sarcastic or negative in tone, and the relatively smaller proportion of posts that were made in disagreement rather than agreement. Do these constitutional forums amount to deliberation? We turn to this question in the discussion below.

Discussion
Our analysis reveals that the Icelandic platform exhibits some deliberative qualities, especially with respect to responsiveness, but it is necessary to consider this finding alongside the complexity of translating normative concepts into empirical measures applied to a case study. We find that, for the threads we studied, responsiveness was quite high, and posters very often consider prior posts before responding. This is what we would expect if the sort of perspective taking and mutual consideration of ideas, identified as essential to effective deliberation (Beauchamp 2020;Mercier & Landemore 2012;Muradova 2021), was happening. To the degree that deliberation is focused exchange of ideas, then, the threads we focused on were deliberative. How might the context of the surrounding constitutional process have mattered?
Certainly, some of the deliberative qualities can be attributed to notable features present in the Icelandic CCPP but missing from many other forums. First, the Icelandic process was, by design, oriented toward initial posts inviting comment. For each thread, there was a clear topic of discussion that provided information for others to respond to and set the tone of the exchange. Public input was prompted with the help of short submissions that contained proposals from interested parties (a number of individuals were contacted directly for the purpose of writing a brief opinion article that would open debate). The drafting committee hoped that conversations prompted by specific topics would guide the article drafting process. 9 Thus, the conversation usually started from a seed that already contained a fair amount of argumentation, which in turn prompted more specificity and argumentation in response. Even a cursory analysis of the 311 proposals shows that they were framed in a way that contained information, argumentation, and a positive valence (Hudson 2018 reaches the same conclusion). For instance, some of the opinion pieces made their case by presenting arguments from prominent philosophers or legal scholars, some incorporated data from the reports of NGOs like the International Institute for Democracy and Electoral Assistance, or referred to international treaties, and a number of them made reference to other constitutions like that of the US, Denmark, Norway, and Sweden. Starting each thread with a model of deliberative communication, within the overall context of public involvement in drafting the constitution, is a structural advantage not all online forums can boast.
One ambiguity in our study has to do with the background of participants. Because this type of online consultation has the form of a SLOP (self-selected listener participation)-or in our case a SROP (self-selected reader opinion poll)-participant profiles are important from the perspective of representation. While they are not a random sample, they may nevertheless represent public opinion fairly accurately if the overall profile of participants matches that of the general public. Yet, as Fishkin notes, SLOPs are usually populated by individuals who feel strongly about a topic, which produces a ' distorted picture of public opinion ' (2018: 22), in the same vein observed in face-to-face deliberation (Verba et al. 1993;Albrecht 2006). In the Icelandic case, we can consider both who submitted the original 311 opinion pieces/proposals and who engaged in the subsequent conversation. The nine most active participants in the proposal portion were male, submitted 24 per cent of the proposals; only 204 individuals were responsible for all 311 proposals. As Hudson shows, a previous study estimates that 80 per cent of the forum participants were 40-65 years old (see Helgadottir 2014). Further, as Katrin Oddsdottir notes, 'the people who participated in the online dialogue were a self-selecting cohort, that is generally more invested in topics such as freedom of speech and the Internet than the members of the general Icelandic public ' (2014: 217.) While we ultimately do not have the private participant data to render a clear judgement on participant interests, there are indications that participants felt very strongly about issues like separation of church and state, a national church, and housing rights. Not only were the threads about these issues uniquely long with many contributors, but they also tended to stay on topic even when posts were negative or sarcastic. We also can infer that after the previous rounds of debates that were broadly popularized (like the two National forums), most Icelanders were generally familiar with the discussion and had a chance to get involved. There was also some incentive to participate, given that it was widely known that the Constitutional Council was reading the comments directly and that Constitutional Council members responded frequently to posts. We also have clues that the participants were highly educated, which is not unusual for Iceland, but it is interesting to note that the features of the discussion with a negative valence tended to be different than that in our previous study in a mid-size city in the US. On the Icelandic forum, negative valence came frequently from a high frequency of sarcastic comments, which we noted in the section above, while in our previous study negative valence was more straightforwardly associated with typical polemic language, denigration, or insults. Sarcasm presents a particular challenge for forum moderators because it is practically impossible to screen through computerized means, but it may still discourage discussion and thus affect the deliberative potential of a discussion. In terms of normative theory about deliberation, future research should consider the degree to which tone in general, and sarcasm in particular, may override elements such as reason and perspective taking.
As we have noted above, our findings must also be interpreted within the sequencing of deliberative steps in the overall process in Iceland: the online platform happened after two national forums, which actively encouraged broad participation. This might have had a positive overall impact on the degree of participation, as well as to the extent to which participants were informed about issues and capable of effective deliberation. The topic of the Icelandic constitution was very much in the public eye, and it was specifically addressing a political problem (the content of the constitution). In many online forums, the topics are ad hoc, prompted by any type of interest, and consequently may range from discussions about public dog parks, to the number of deer in urban backyards, to the school board elections and other education issues. In this regard, the incentive to arrive at a clear outcome by reading and responding may have been strengthened because the value of the process was regularly reinforced by the external processes of constitution making.
But even with a high degree of prompting coming from experts or well-informed civil society leaders, the Icelandic forum exhibited a surprisingly low level of overall participation. As we remarked above, 145 out of the 311 threads had two posts or less, and an overwhelming 287 had less than 10 posts. Many well-argued points remained unanswered and only the three most contentious of issues attracted significant debate. It is not surprising that contentious issues generated much more debate, but the very low response rate for other issues requires us to ask questions about the deliberative qualities of the forums overall. If only very few issues generated interest, then a large number of other constitutional issues were ultimately decided by the small group of people on the Constitutional Council. This invites us to reflect on the set-up of a complex process and a sort of issue-fatigue: the public might use its naturally limited attention and time on the few issues it considers critical while putting others aside. This might be a reason to compartmentalize issues rather than embarking on one massive project like a constitution and cover everything in fast sequence. There was certainly a time constraint in the Icelandic case, since the Constitutional Council was only afforded months to write the draft and consult the public, but this seems to have set the process down a path of issue-fatigue and selective attention. This in turn has the potential of shifting the drafting responsibility to a few people and increasing their imprint substantially, even at deliberative steps intended to increase public participation. More optimistically, future implementation of online deliberation may learn from the Icelandic model to limit discussion to the most contentious issues while also establishing a structure and tone that favors reasoned engagement with others' perspective and ideas.
Here we note that while this was called a ' crowdsourced constitution' draft, the public input was much more limited than typical crowdsourcing (Suteu 2015) and certainly more than what can be found on typical online forums. In the words of the constitutional council, the process of consulting the public was more accurately described as ' an exercise in open democracy and transparency rather than crowdsourcing' (Valtysson 2014: 63). The set-up for the public input was nested in a sequence of steps that each shaped the ultimate outcome. For instance, the Constitutional Committee also had access to other experts, mostly lawyers, who could craft the phrasing. But as Suteu remarks, experts had ' a significant impact on the drafting process' as their work sometimes ' exceeded mere wording' despite some objections and ' amounted to an alteration of the substance and meaning of certain provisions' (Suteu 2015). Many of these alterations were executed with an eye to conform with international treaties and legal regulations emanating from the EU, 10 but this step was at odds with a deeply deliberative setup.
So, given these contextual factors, what can our analysis tell us about the prospects of online deliberation? On the whole, Iceland's use of online deliberation as part of a larger multistage process was relatively successful. From the perspective of effective deliberation within this stage, we conclude that the online portion of the process generally effective and mostly free of the negativity and trolling common online (Coe, Kenski & Rains 2014). Of the three threads we examined, most posts were neutral to positive, and while many were critical of prior posts, they were responsive to the content of the posts rather than off-topic or attacking the original poster. This serves as evidence in support of claims about the deliberative potential of the internet in general. However, it must be noted that these online conversations were in the context of a larger deliberative process, and though the CCPP used a Facebook plugin, it was quite clearly demarcated from the typical Facebook space. Overall, Facebook has not proven to be a space conducive to rational discourse and decision making, and there is evidence to the contrary, that it may be fostering anti-democratic movements (Vaidhyanathan 2018). The context here matters, because the forum was on a dedicated website marked as a government platform (.is). It is reasonable to conclude that the official platform shaped expectations regarding the purpose of the discussion and the intended audience. In other words, the tool (Facebook plugin) was less important than the context and its use. The fact that comments were prompted by a specific informative piece was also useful, as it discouraged discussants from introducing wildly disparate topics. The presence of a moderator together with a degree of input from the Constitutional Council members themselves seem to have also contributed to a higher ' deliberative stance' (Owen & Smith 2015) than usual. In other words, the on-topic quality of most posts could be attributed to a setup which created what Owens and Smith describe as ' a relation to others as equals engaged in mutual exchange of reasons oriented as if to reaching a shared practical judgement' (2015: 228). Whatever deliberation did occur, it did not happen in a vacuum, nor in a purely online space. External factors set valuable constraints, and participants were likely uniquely committed to deliberation. As scholars begin to note how features of web design affect online social processes, they must also note how external political and social structures can add to, or detract from, deliberative exchange.
The flip side of this conclusion is that the deliberative stance was mostly built on a very homogeneous public of highly educated Icelanders, and ultimately, not that many people participated. It is difficult to disambiguate the role of homogeneity without detailed demographic data, but the baseline itself (Iceland) does not present a diverse group. Even so, the Icelandic context projected the idea of a process in which civic participation is part and parcel of the drafting practice 'in order for a constitution to become a vibrant reflection of a political community's political imagery and self-understanding' (Bergsson & Blokker 2014: 154). In other words, the bottom-up approach validated public purchase into the process and restored the constitution to those from which it is supposed to emanate: the general public. Mitigating the role of elites in the drafting process strengthened a democratic element which, as some scholars point out, has been sorely lacking from modern constitution-making processes (see Lessig 2012).
Overall, the empirical evidence here supports cautious conclusions regarding the possibilities of an intentional deliberative setup that is hosted on a legitimate site, is moderated, non-anonymous, guided by specific topics, and encourages the direct involvement of policy makers. We also note the similarity of our findings to recent studies of successful online deliberation (Cagle and Herndl 2019;Curato et al 2017;Eisenstadt and Maboudi 2019). A consensus may be emerging: online deliberation is possible when the parameters of discussion are well defined and enforced, topics are clearly stated, and the process is inclusive by design.

Notes
1 The Constitutional Council official website hosted the forum which can be found at: http://www. stjornlagarad.is/erindi/ The authors would like to thank Eileen Jerrett (documentary filmmaker and independent researcher) and Finnur Magnusson (CTO of the Icelandic Constitutional Council) for their assistance in providing the complete archived data of the public platform. For questions about data availability, readers should consult with the Constitutional Council via http://www.stjornlagarad. is/upplysingar/#hafasamband. 2 Please note that some texts/authors use the phrase 'National Assembly' (see Gylfason 2011).