Anouk Lang: Developing methods for analysing and evaluating literary engagement in digital contexts

This project, Developing methods for analysing and evaluating literary engagement in digital contexts, begins from the starting point that the rapid rise in the amount of user-generated content produced on the internet, especially on social media sites, offers an extraordinary opportunity to study human interaction in a format that lends itself easily to multiple kinds of computational analysis. From the perspective of scholars of reading and reception, this growing body of data is particularly exciting, given that it is not just time-consuming to interview individual readers, carry out surveys and conduct focus groups, but also problematic to draw conclusions from artificial contexts where it is difficult to know the extent to which the answers being given have been influenced by the unequal relationship between reader and researcher. Although data derived from the internet has plenty of limitations of its own—the fact that users of a particular site or service may not be a very representative sample of the general population, for instance—it is still the case that born-digital responses to texts, other readers, and literary events offer researchers the tantalising possibility of grasping aspects of reading that have been previously inaccessible. Not only are there much larger amounts of material available than in the past, but also digital reception data often involves readers voluntarily recording their thoughts in the context of a community to which they feel a sense of belonging, rather than reporting them to a stranger.

The challenge for researchers who work on reading and who do not have large amounts of technical background knowledge is twofold. First, how can they access these rich bodies of data, and second, how can they carry out analysis of digital materials alongside their established methods of working with non-digital reception data? A scholar with experience in interpreting marginalia – comments written in the margins of books – is well placed to bring her skills to bear on digital forms of annotations, for instance, but might not know how to get hold of this data nor how to process it when the sheer amount of material available exceeds the capacity of a single human reader. Other disciplines have addressed these issues—corpus linguists have established methods of constructing and analysing large textual corpora, for instance, and computer scientists have developed techniques such as sentiment analysis which can process large numbers of statements to determine whether they are broadly positive or negative, while various other approaches are being taken by scholars across the digital humanities—but for scholars of reading without the technical background to scrape data from websites, or set up a Twitter archive, there are significant barriers to engaging with this data.

The aim of this project is to lower these barriers, by reporting on three different kinds of approaches that can be taken with digital reception data that are within the grasp of reception researchers without specialist digital humanities training. First, it examines the thematic content of the textual data that individuals generate when they engage in online discussions about the value of books or literary activities. Second, it investigates what can be learnt from the chronological information attached to these discussions, for example the timestamps on social network posts or tweets. Third, it considers the role played by place in online conversations about reading, using digital mapping tools to visualize the geographic information attached to social media posts. The project will produce a report setting out what kinds of information can be learnt about the cultural value of reading in the digital age from these three angles, and will supply guides for a number of digital tools which can be used to work with these three kinds of data.

The two types of social media on which the project centres are the micro-blogging service Twitter and the literary social network LibraryThing. Because the focus of the project is the value that reading and book-related activities brings to individuals, I have chosen books and authors that have won or been shortlisted for high-profile prizes such as the Nobel Prize and the Booker Prize, and that have featured in literary competitions with considerable cultural cachet. Using timestamped data from the Twitter API, for instance, will allow me to examine such things as how the content of discussions about a shortlisted book change in light of prize announcements, or how the progress of a literary competition might influence the way LibraryThing users position themselves in relation to a particular book as they go about their interactions with other readers on the site. Geography, too, can be considered: as people across a country or around the world take to Twitter to express their opinion about an author who has just won a prize or a competition, what kinds of patterns is it possible to discern from the spatial distribution of tweets? Previously, it was difficult for scholars of reading to access the when and where of reception data with such precision, and so—especially in light of the large amount of material that is now available online about readers’ preferences and responses to books—it seems an opportune moment to reflect on the methodological opportunities and limitations of this kind of digital work on the cultural value of reading.

Eric Jensen: The Role of Technology in Evaluating Cultural Value

The breadth of artistic and cultural practices that may connect to the development of cultural value has so far extended well beyond the reach of existing methodological frameworks and research methods for systematically capturing, analysing and accurately representing the different components of these impacts.

This critical review project will employ a rigorous cross-disciplinary, analytical literature and methodological review to identify and evaluate the different ways in which technology can be used to evaluate different components of cultural value. A wide variety of settings will be addressed such as arts and culture museums, arts and culture festivals, cultural events, local and national arts programmes, etc. In this project, my expertise in technology-enhanced impact evaluation is supplemented by a leading scholar in the role of technology in arts and culture, Professor Mike Phillips at i-DAT (Plymouth University). This desk-based research project addresses the uses, quality and possibilities of technology-linked evaluation methodologies and strategies, which could be used to help evaluate the development of cultural value in different settings.

This project will address the following research objectives:

1. Develop a catalogue of current uses of technology-enhanced evaluation methods and approaches to measure cultural value and related impacts.
2. Critically review the strengths and weaknesses of different technologies for evaluating different components and dimensions of cultural value in a range of contexts.
3. Develop a report describing different technology-enhanced methods of evaluating cultural value, including a master table of methods, technologies and approaches; along with a critical assessment of their strengths/weaknesses.

This research contributes to the Cultural Value project by informing the evaluation of cultural impact to support robust, long-term, empirical research on cultural value. Meanwhile, technological innovations have raised new possibilities for evaluating cultural value that promise to increase efficiency, reach and validity beyond that which is possible with conventional methods. This proposed project considers several technology-enhanced methods for evaluating cultural value, providing descriptions and critical assessments to elucidate their strengths and weaknesses. This project focuses on evaluation methodology per se, with the aim of supporting future empirical evaluation and research on cultural value. This project will review current literature on the use of technologies to evaluate the development of cultural value through events, institutions and digital platforms such as Twitter, Facebook, individual webpages, etc.

The project team will identify the strengths and weaknesses of different technologies and technology-enhanced evaluation approaches to suggest an appropriate framework for making best use of digital technologies in cultural value evaluation. The results of this critical literature review can be broadly applied to a framework that can advance how the value of cultural engagement is evaluated and employed.

This project will integrate a range of research approaches, embracing the partners’ knowledge of arts/culture engagement impact evaluation; software design; production and implementation; mixed methods data analysis; visualisation of digital programmes; and the requirements for integrating these new modes with more traditional cultural impact analysis. At its core, the project is focused on how we can capitalise on digital technologies to build a robust, evidence-based evaluation framework; while at the same time maintaining a critical perspective regarding the limits of the technologies’ contribution to evaluating cultural value.


Harriet Hawkins: Experimental Methods for Exploring Environmental Encounters

It is well recognized that a host of aesthetic strategies – from artistic practice to visual culture more broadly –respond to, and often move for action in the face of, environmental change and current ecological crises.

Environmental art, as examples developed by the AHRC funded Landscape and Environment Programme demonstrate, is wide-ranging; encompassing different mediums, (e.g. performance and body art; participatory art, story-telling) themes, (e.g. local myth and lore, pollution, imaginative and exploratory engagements with scientific data-sets) and philosophical positions (e.g. encounters with animate earth-matters).

Despite the popularity of these art forms, and the recognition of their inter-disciplinary value within and beyond the academy, it is equally well recognized that we lack an understanding of the nature and importance of the environmental encounters that these works catalyze, as well as those encounters catalyzed by the curation and programming associated with these art works.
This research has set out to explore the environmental encounters configured by arts projects. It has focused on projects developed by two art-science organizations, Arts Catalyst, a project-based art-science commissioning and curatorial agency located in London, and Swiss artists-in-labs (ail), an art-science residency programme based in Zurich.

The challenge: researching encounters- theorizing encounters

Previous research I was involved in on the geographies of art-science collaborations highlighted two issues that needed further engagement, issues that the “Experimental Methods” project took up.

Firstly while the earlier project studied how art-science collaborations transformed the artists and scientists involved, re-shaping their knowledge making practices, and reorganizing relationships between individuals and technologies, what we did not explore where the effects of the resulting art work on audiences. The need for this information on audience experience was reinforced in the course of conversations with the international organizations we collaborated with on this project. These organizations wanted an evidence base that offered reflections on the strengths and weaknesses of their practices, this evidence would enable them to develop their practice, but also to underpin claims made to public and private funders in the art and science worlds alike.

Secondly, one of the reasons the previous project had not been able to develop effective material on audience experiences related to the lack of a coherent and appropriate set of methods by which to research audiences and their experiences. As a result, in order to develop the evidence base noted above, what was needed was more foundational work on the research methods that would enable the study of audience encounters with environmental art works.

At the heart of the research project that evolved therefore sits a key tension, namely that while a diverse array of philosophical frameworks have formed the means for conceptualizing arts’ ‘environmental encounters,’ what we often lack are methodological discussions and evidence bases to complement these abstract theoretical modes, thereby extending examinations of these encounters and their transformative potential.

The “Experimental Methods” project responded by setting out to explore what kind of methodology, and what sorts of research methods, might be appropriate for researching the environmental encounters that are catalyzed by environmental art works, whether they be embodied experiences of the environment, assertions of dynamic earthly matters, and atmospheric ‘airy’ materialities, or the creation of ‘radical publics’ through participatory arts practices.

Research is focused through three key questions:

1) What are the forms and experiences of the environmental encounters configured by art projects?
2) What ideas of ‘evidence’ and ‘evaluation’ are appropriate for exploring such encounters?
3) What kinds of methods can help us to engage with these environmental encounters?

In exploring these questions “Experimental Methods” will engage with some of the key aims of the Cultural Value Project, namely, developing both conceptualizations of cultural experiences –‘reopening the question of what engagement with cultural activity does for people’– and experimenting with innovative methods by which we can understand and evaluate these experiences. In addition to the project report, outputs will include a series of project seminars aimed at arts organizations as well as academics, it will also include academic papers given at conferences, and an edited collection on “Geoaesthetics: arts and environmental encounters”

Harriet Hawkins is a senior lecturer in the Department of Geography at Royal Holloway, University of London. She is the PI on the nine month “Experimental Methods for Engaging Environmental Encounters” project. She is author of Creative Geographies: Geography, Art and the Making of Worlds (Routledge, 2013), which introduces some of the work that underpins this research project.

Sally Munt: Cultural Values from the Subaltern Perspective: A Phenomenology of Refugees’ Experience of British Cultural Values

This project seeks to understand the value located in a range of arts/cultural activities to refugees, asylum seekers and migrants, a group new to British cultural life who are often marginalised from ‘mainstream’ cultural activities, but who are simultaneously expected to adopt a hegemonic national identity of Britishness and henceforward espouse British cultural values. Refugees are a group who typically have experienced forced migration, oftentimes related specifically to their own – often fiercely defended – cultural activities and values in their country of origin. This migratory biography makes for a complex, rich contribution to how we think about the value of arts and culture, and cultural expression, in the UK today.

We will investigate the standpoint of refugees, asylum seekers and migrants on British cultural values, benefitting from their ‘outsider within’ perspective.

British cultural values are not unitary, nor are they precisely definable, they are shaped and refined by participation and engagement. We will seek to identify the components of cultural value embedded in a set of typically British arts and cultural pursuits, based in and around the city of Brighton.

We will break down the components to be identified using a range of methods that focus on the discrete senses, and on the particular forms of embodiment that such activities claim. We want to examine carefully what constitutes the experience of involvement in the arts and cultural sphere, so we will also be collecting information on the cognitions and emotions that are attached to such experiences.

Refugees constitute a unique case: migrants pay acute attention to the acculturation of British values. This attention can be a protective mechanism, a philosophical choice, an attempt to move away from a traumatized past or culture of origin, an imposed set of norms, or a way of making their enforced dislocation intelligible. Refugees are legally required to learn British cultural values in order to be ‘awarded’ citizenship, via the Home Office instrument, the ‘Life in the UK’ Test (which will be interrogated in group discussion). Whatever the reason, refugees have an acute sensitivity and prescient awareness of ‘what makes us British’. Yet, often their access to the cultural industries can be severely restricted, due to explicit factors such as economic barriers, and due to implicit factors such as the perceived ‘Whiteness’ of some art/cultural pursuits (eg. premier league football, and the opera – two performances that will form part of our programme).

This project will take the form of a 16 week course, called ‘What is British Culture’, offered to 14 women refugees. Through a range of arts and cultural activities, we will assess refugee’s embodied experience of participation and reflection, gathering sensory information through creative expression. In order to gather robust data, the course is quite long and demanding; however we have found in previous projects that refugee participants appreciate such commitments as they enable a strong group identity to form, which can continue informally after the planned meetings finish, providing a sustainable resource.

As researchers we have our own cultural values: our model is taken from feminist praxis. Feminist epistemologies focus on the way “in which gender does and ought to influence our conceptions of knowledge, the knowing subject, and practices of inquiry and justification” (Anderson 2004). At the core of feminist epistemology is the concept of the situated knower, who produces situated knowledge. Donna Haraway (1998) famously argued that most knowledge, in particular academic knowledge is always “produced by positioned actors working in/between all kinds of locations”. Collaborative learning, respect for social difference, creating an environment of mutual support, listening and consideration for others, these characteristics are all markers of the feminist classroom, cultural values which we hope to emulate in the process of the research.

We are now two thirds of the way through the project and have recruited 14 women from 9 different countries including Sudan, Iran, Iraq, India, Pakistan, Egypt, Palestine, Zimbabwe and Ghana. Attendance has been strong, and we have completed a range of activities including visits to Brighton Royal Pavilion, Brighton Jubilee Library, Brighton Museum, Preston Manor, a seaside walk on the seafront, yoga and meditation, and life history exercises, and are looking forward to watching England womens football team play Montenegro live at the Albion Stadium, and attending Onegin at Glyndebourne Opera House. We have completed individual interviews, focus groups, and ten class meetings. We look forward to exploring our findings.

Gareth Hoskins: Locating value: assigning significance in the historical built environment, a trans-Atlantic review

All places are valuable but some are more explicitly valuable than others. Who decides which are most precious and most worthy of public esteem, and how do they do it? My ‘locating value’ project investigated US and UK preservation agency practices of listing, landmarking and designation. Visits to Washington DC and Bristol allowed me to examine the development and application of the various criteria and protocols of assessment employed in the evaluation of historic buildings.  My aim was to use historic preservation as a working context through which to test the age old philosophical question: is value found or is it made? I wanted, specifically, to remove some of the mystique that surrounds the assessment of non-economic kinds of value, and to challenge the veneer of academic objectivity that gives such systems their authority.


In the end I encountered a group of dedicated practitioners in the awkward position of both believing and not believing in the fair and ‘neutral’ valuational frameworks they work to apply on a day to day basis. On the one hand preservation officers could use their system to defend themselves against charges of bias: “We don’t have views, we’re not trying to be arbiters – we feed nominations into the system and see how they measure up”. On the other hand many recognised that influence was often and is still brought to bare in the composition of lists either through external political pressure or internally held personal preferences and priorities.  Less obvious though was the systematic bias coded in to logics of selection that privilege the kinds of histories that are already heavily represented.


It was a tremendous conceit for the originators of the lists to assume that their own tastes and sensibilities (those of a white male professional class patronised by an economic and cultural elite) could operate as the default setting of value and work on behalf of us all.  Today the skewed nature of the United States National Register of Historic Places, the National Historic Landmarks Program, and the UK’s various statutory lists of buildings, serve to record, amongst other things, political influence. Those populations underserved by preservation agencies in the past are now being specifically targeted with outreach initiatives and an associated effort to expand lists in directions that better reflect social and ethnical diversity.

The research project developed a relational theory of value to question this kind of ‘inclusion via expansion’-based solution.  A relation theory of value asserts that any judgment of value involves the reciprocal removal of value from something else. We are not simply saying therefore that one thing is good, we are unavoidably saying that one thing is better than another, one building should be ranked higher than another, one application for funding is more worthy than another, and so on. Value relies on creating an equivalence. It operates in a zero sum game. Value cannot be self-sufficient and infinitely amassed since every attribution leads also to a disavowal.  By identifying only the very best and showcasing the exclusively positive outcomes of their listing practices preservation agencies in the US and UK ignore value’s limits.  In his essay on the Destiny of Value Baudrillard makes this same point: “Because we no longer know what is true or false, what is good or evil, what has value or does not, we are forced to store everything, record everything, conserve everything, and from this an irrevocable devaluation ensues… all that lives by value will perish by equivalence” (1998, 4).  So if value as a concept is to promote social justice and advance equality then what we require more than anything else is its re-distribution. We can’t simply add more things to a list, we need to take things off the list as well.