Monthly Archives: May 2015

In Defence of Star Ratings

Last Autumn (too long ago, sorry Chris) – I had a conversation with Chris Goode on Twitter about the Albany’s feedback forms, prompted by his tweet:

‘Bit sad… to see brilliant @TheAlbanySE8 soliciting star ratings from audiences on its otherwise interestingly nuance feedback forms…’

His comment related to a specific question on Albany audience feedback forms which asked:

‘if you were reviewing this production for tomorrow’s papers, how many stars would you give it?’

This question sat alongside a number of ‘differentials’, asking audiences to rate their night based on five dimensions of the audience experience: 1) engagement and concentration 2) learning and challenge 3) energy and tension 4) shared experience and atmosphere 5) personal resonance and emotional connection.

He asked if the data gathered from the star ratings informed decisions about which artists get supported/asked back, and pointed out that the question about how star ratings are used in feedback forms is an adjunct to a live conversation about how star ratings are used more generally to critique work, particularly in media reviews.

Andy Field, artistic director of Forest Fringe, had recently written:

“Star ratings are the very worst – nuance’s kryptonite – tiny nuggets of sadness harvested in the darkest heart of consumer capitalism and sent to cling grimly to the surface of art like old shopping bags floating down a river.”

Which seemed to summarise the general mood in the alternative arts world about star ratings fairly neatly. He wrote:

“In (using star ratings) we’re helping to perpetuate a thinness to the way in which people can engage with our work, and the means they have of navigating their way through it. To an extent, the very stars that we use to attract people are actually perhaps limiting the number of people who will engage with our work and the ways that they have of engaging with it. It’s a vicious circle, but the point is, we are part of that circle, not simply it’s victims.”

I remained somewhat more ambivalent on the matter. While I could recognise the frustrations some artists felt about the reductive nature of critiquing work via a star system, I thought in the context of our feedback forms – where we ask our audiences for their perspectives in a range of ways, not simply via a star rating – they might be ok – and even useful, to both venue and artists alike. But I hadn’t written our feedback forms (they predated my time at the Albany), so being the thorough professional I am I decided to undertake a deeper exploration of perceptions of feedback forms, and review our current feedback forms based on this.

Here’s what I did. I undertook a survey of Albany staff as well as artists and producers we work with regularly, garnering their views on our current feedback forms, and how they use them in their decision making processes.

I reviewed the literature around approaches to gathering feedback from audiences. There’s not much around. Matt Truman wrote an interesting blog about feedback processes here which called on venues to reject feedback forms, seeking opportunities for more nuanced, direct conversations with audiences, particularly in the context of works still in development, where audience feedback is likely to have an impact on the direction of the work.

The most significant, and comprehensive, review of how feedback is gathered remains that published by ITC (Independent Theatre Council) way back in 2005. The publication outlines research undertaken by the New Economics Foundation on ‘capturing the audience experience’. It offers an in-depth analysis of how feedback can meaningfully point to the impact of an artistic experience on the general wellbeing of an audience member. It is from this report that the questions on our own feedback form are drawn, including that thorny star rating. In the report, the star rating is not designed to, in and of itself, offer a measure of the value of a particular experience – but to be used in reference to a number of other evaluation questions, based on different areas of audience perception, to establish what aspects of an artistic experience lend it value from the point of view of an audience.

I also co-hosted an event with Jake Orr and Maddy Costa exploring the relationship between critics, marketers and audiences, where a discussion took place regarding the use of star ratings. Maddy summarised the event here. While a range of views were expressed regarding star ratings, I found myself in the naughty corner with a bunch of arts professionals who thought that, after all, star ratings might be a simple metric that’s comprehensible to everyone.

On the strength of this period of investigation, I reviewed our feedback forms, and reprinted them with some slight changes. Perhaps the most significant was moving the star rating to the bottom of the form (below the audience experience differentials outlined above), with the intention of encouraging audiences to think through the different aspects of their experience before offering a star rating. (A link to the new feedback form appears at the bottom of this article). We have also reviewed the way data sources are drawn on in the decision making process at the Albany, and how this information is shared with artists.

Nothing that emerged through this investigation convinces me that the negative impact of star ratings is such that it outweighs the many ways in which they are useful.

We’re used to rating our consumer experiences with stars in modern society – on Ebay and Tripadvisor – and perhaps this is what Andy means when he writes they are ‘harvested in the darkest heart of consumer capitalism’. But a more optimistic view might be that the invitation to ‘rate’ an experience makes us more proactive and engaged as consumers. Star ratings are by their nature an oversimplification; but the benefit of simple evaluation tools is that they are accessible. Star ratings offer a means of ‘taking the temperature’ of our activity evenly across the season, with the broadest possible cross section of audiences – not only those that have the confidence to articulate complex perspectives.

Clearly this needs to sit alongside a more nuanced programme of engagement, and the Albany’s commitment to drawing audiences into the creation of work from the very beginning is a clear example of how this happens here (recent examples include Ampersand Media’s Storylines, a monthly performance created with a newspaper reading group; and Simon Mole’s No More Worries, on which he’s run workshops with community groups during its creation), as well as, for example, the chalk board we place outside the theatre for audience to share their immediate response to performances, or regular post show discussions.

But the consistent information we’re able to get from feedback forms has an important role to play. It can be a departure point for a conversation – why do certain shows appeal to our audiences more than others? Do certain shows resonate with particular demographic groups, or work better at one venue than another – and if so, why? As with all evaluation tools, we use them as a way to reflect on the decisions we’ve taken and the impact they’ve had, rather than as a means of assessing ‘success’ or a ‘failure’.

Audiences, incidentally, are generally far more generous in their response than critics. That disparity is interesting – in my view it reflects the fact that the majority of professional theatre criticism emerges from a pretty narrow cross section of world views – and can be a means of making an argument for the value of a piece of work that doesn’t necessarily sate the appetite of the mainstream media.

The best evaluation synthesises information from a number of different sources, and interprets it through the lens of human intelligence. Quantitative data – for example star ratings – are one source of information. Qualitative data – for example personal comments and reflections gathered from audiences via feedback forms, social media, and good old-fashioned face-to-face conversations, are another. But in none of the organisations in which I have worked has either of these sources of information outweighed the importance of the personal understanding we have of the artist’s creative aims, and how these marry with what we consider to be urgent questions and approaches that are relevant to the contexts in which we operate. Nor are strong leaders of arts organisations shy of programming or producing work that may be more challenging for audiences – in fact doing so can often be the most powerful decision they can make.

You can view our new feedback for here.

Amber Massie-Blomfield,Head of Communications, The Albany

1 Comment

Filed under Uncategorized

Meet Me at the Finish Line

ben lunge croppedMeet Me at the Albany, our pioneering programme for older people, has been going on for two years now, which barely feels like any time at all given that it is such an integral part of our programme and building. In a bid to help with the funding of the project, our Head of Production & Premises, Ben Stephen, has decided to get off the couch, get into shape and run the Bupa 10k run on 25 May, all to support Meet Me at the Albany. Ben lets us know about his motivations and also his training before the big day:

Why would a middle-aged, fat, technician run 10km?

To me this seems a perfectly valid question.  I actively dislike exercise.  I have survived in my job roles and life by balancing desk work with a bit of throwing steeldeck around, running up and down ladders to keep me moving and shaking, and justifying a bacon sandwich for breakfast and a burger for lunch.  The trouble with getting older for me is that you start to think more.

I’ve been working at the Albany for a couple of years now and together with Entelechy there’s this amazing project called Meet Me at The Albany which started just after I did.  The project is based on a really simple concept – life is better when there are people, activities and arts in it.  For older people isolation is an all too real possibility.  Failing health, the loss of a partner, absent children or just plain lack of contact with other human beings is a very real scenario and it is a downwards spiral if you’re trapped in it.  Us “young ones”, as we’re referred to, can seemingly account for this with the use of social media or just going and hanging out on the street corner.  If you’re over 60 (admittedly this is an arbitrary number) then you tend to feel the cold a bit more, find things harder to do potentially, but no one seems to think it’s time to go on the scrap heap just because you’re older.

Meet Me sticks two rude knitting needles up at this.   Every Tuesday the volunteers arrive (a significant proportion of them over 60 and feeling the importance of facilitating activities for their peers) meet up with some Albany staff members and get the tables all moved round, we chalk up the day’s activities on a board, get the tea and coffee on the go and meet the artists who will direct the artistic output.  Then the participants start rolling in from their minibuses, walks down through the estate and buses from their homes.  There’s a significant and constant creation of visual arts including painting and sculpture, weaving with huge ropes, knitting Christmas decorations and cat toys to sell.  They take part in singing workshops, trumpet lessons, creative writing and theatre watching with some really difficult questions for performers in special Q&A’s.  One of the members is making a full length animated musical about living in a nursing home on his laptop during the tea breaks, using the stories and ideas around him for inspiration.  There have been circus skills workshops and poetry slams over Skype.  Have you ever seen someone using a walker balancing a peacock feather on the end of their shoe?  It makes you challenge your prejudices and comfort zones.

On a terrible day 15 seconds in the company of Meet Me At participants and volunteers will fill you with sunshine and renewed faith in humanity.  It is awesome and it’s addictive and it’s a vital social lifeline to the people who come to it!

It costs a lot of money to run this project successfully. So taking the Christmas decorations as inspiration I thought ‘what can I do?’  I could try and run around central London in a bid to get some financial support for this amazing social enterprise.

So I signed up to the Bupa 10k, on 25 May (Bank Holiday Monday) starting on the Mall at 10am.

I started training for this in secret before Christmas.  I discovered I couldn’t run for the bus without feeling a sense of impending doom.  So I started off with the couch to 5k stuff (a minute running, a minute walking) and slowly started to get there.  I feel I have a solid 5k now and I’ve been trying to lengthen this without injury or other disaster.  I busted my knee up pretty badly in February trying to get up to a half marathon distance so I’ve wound it back and I’m aiming for a solid 10 kilometres in a sensible time where I can walk the following day.  So far 8.5k near killed me on Sunday.

Watch this space for training updates, there’s three weeks to go and I’m at 25% of my fundraising target…..

Ben Stephen, Head of Production & Premises, The Albany

To help sponsor Ben’s run towards the funding of Meet Me at the Albany, click here. For more information on the Meet Me at the Albany programme, visit their website here.

Have a look at Ben’s workout routine in the video below:

1 Comment

Filed under Uncategorized