The long-awaited Social Housing White Paper was published in November and Housemark was pleased to see the new charter for social housing has residents at its centre.
The long-awaited Social Housing White Paper was published in November and Housemark was pleased to see the new charter for social housing has residents at its centre. The charter aims to reset the relationship between tenant and landlord based on evidence, transparency and trust. Key to this transformation is the inclusion of customer satisfaction (CSAT) metrics that landlords will have to report against.
One of the largest social housing providers in the UK recently asked me for help setting customer satisfaction (CSAT) targets. They wanted to make sure the targets were ambitious and got to the heart of what matters to their residents, but appropriate to their context.
And that’s no surprise. At Housemark we know from the CSAT data we hold for over 400 landlords going back over 10 years that performance is heavily influenced by context. When the ministry was consulting on the consumer metrics that now feature in the white paper, over 10% of respondents expressed concern about understanding the context behind performance measures.
How does context affect satisfaction scores?
The age and ethnicity profile of residents, their tenure type, where they live and the condition of their home can all have a significant impact on how satisfied people are likely to be. These ‘contextual variables’ can make comparing more difficult.
At Housemark, we have been helping our members understand contextual variables for many years. Performance against all sorts of metrics and even unit costs are often impacted by the context within which the landlord operates.
Most recently, our monthly COVID-19 impact monitoring has helped landlords understand the impact of the tiered system of lockdown – another contextual variable impacting key measures like arrears and repairs.
But when it comes to satisfaction metrics, contextual variables aren’t the only factor affecting performance.
Its not just what you do – it’s how and when you do it too
When our client looked at other landlords operating within a very similar context, they also noticed huge differences between reported CSAT results (ranging from 60% to 95%) – these can’t be explained by context alone.
In fact, many of these large landlords are collecting customer feedback in different ways which can produce profoundly different results – what we call in the industry ‘methodological variables’.
Chief among them is where landlords collect customer satisfaction through transactional means, rather than a perception survey (which is a random sample of all residents).
Transactional satisfaction surveys are often carried out following a repair, a complaint, a new letting or a call to the contact centre. When responding to transactional surveys, residents are typically thinking about their most recent interaction rather than their overall perception. As a result, transactional surveys typically provide CSAT scores that are around 12% higher than perception survey scores.
On top of this, changing a response scale can also have an impact. Asking for a yes/no response will produce higher results, as does asking for satisfaction on a four-point scale (rather than a five or ten-point scale which allows neutral responses).
The third most significant methodological difference relates to the survey method – online surveys are increasingly carried out by larger landlords in particular (48% of landlords currently collect some or all resident feedback through online channels) but these tend to attract younger and more opinionated respondents which can on balance significantly reduce your CSAT score.
Unravelling this can be difficult for both landlords and their residents. A Housemark survey of over 13,000 residents across 40 social landlords showed that almost half of residents didn’t trust the CSAT scores reported by their landlord, and 43% said they did not believe their landlord would act on customer feedback.
A question of trust?
This issue of transparency and trust is a key theme underpinning the metrics in the social housing white paper. Many landlords are going to have to up their game in how they collect, report and act on customer feedback.
The new STAR framework released earlier this year provides a widely accepted methodology for collecting and comparing CSAT scores. It was built in consultation with over 13,000 residents and 300 landlords with input from key representative and regulatory bodies such as the NHF, CIH, ARCH, NFA, TPAS, TAROE, and the RSH.
It is a valuable tool in helping to restore trust between the landlord and their residents in figures reported.
Tpas were delighted to partner with Housemark by running a series of workshops to ensure our tenant members had a strong voice and influence in the design of the national STAR survey. We know that in the past residents have often been cynical of satisfaction performance reported by landlords. The new STAR framework offers a valuable opportunity for landlords to re-set the relationship, focus on what matters to tenants, and access meaningful insight to help drive improvement.
Jenny Osborne – CEO of TPAS
New tenant satisfaction measures for English housing associations, ALMOs and local authorities are on their way
The core and recommended questions in the STAR framework closely reflect the questions included in the white paper. Given the extensive consultation process with both residents and landlords in building the new STAR framework, it is perhaps no surprise that there are similarities.
Whilst some landlords still prefer to collect satisfaction scores their own way for a variety of reasons, the clock is already ticking on this approach. Over the coming months the regulator will need to propose and consult on definitions that will provide enough consistency for them to effectively regulate.
Strict criteria on things like question wording and order, response scales, collection method and sampling would help drive down methodological variables, increase transparency and provide the regulator with more like-with-like comparisons, although contextual variables will continue to exist.
Given the extensive tenant and sector involvement in building the new STAR framework and its widescale adoption by the sector (over 72% of social landlords in England now carry out STAR surveys on a biennial or rolling basis), Housemark feel confident that we have set the standard. The new framework is built on best practice, is applicable to landlords of all shapes and sizes, and provides a consistent methodology which supports comparison. The STAR framework does however also allow landlords some flexibility in how they get the intelligence they need to drive improvements. Whether or not this flexibility will be retained in the eventual regulatory definitions remains to be seen.
So where does that leave our clients who want to set targets for CSAT?
Well the great news is that Housemark has in-depth knowledge of all the variables that can influence satisfaction scores, and to what extent. We have robust data which enables us to explain how you truly compare, given both your context and your survey method.
With the white paper clearly setting out the direction of travel, landlords need to really understand their CSAT scores now, to avoid any unpleasant surprises in the future. Any large landlord reporting over 90% satisfaction is very likely to see this figure drop when they comply with the methodology that the regulator will in due course provide.
Housemark’s top tips to ensure you are ready for CSAT regulation:
1. Interrogate the scores you are currently seeing. What do residents say about them? Do they really reflect the services you provide?
2. Contact Housemark for insight into what variables are affecting your scores. This advisory service is provided free to our members and will help you understand how you really compare. Email data@housemark.co.uk
3. Review your methodology to ensure it includes the proposed regulatory questions and is compliant with STAR framework guidelines on collection method, response scales, response options and sampling – this will ensure there are no nasty surprises when regulation arrives.
4. Keep surveys short to minimise survey fatigue. Take an organisation-wide approach to collecting feedback so that the same residents aren’t repeatedly surveyed by different departments. This also helps to triangulate what your residents really think.
5. Be clear on what you do with the data. If you’re collecting information you don’t use, then drop it.
6. Invest in your data analysis capability (whether this is in-house or outsourced). This will ensure you really understand how you compare and can turn the data into the insight you need to improve. Sentiment analysis based on free text responses can be incredibly powerful but requires resource, skills and commitment to do properly. If you need help, the data specialists at HouseMark are on hand.
7. Get underneath your data – consider health checks and accreditations for your key resident services, in areas such as complaints aligned to the Ombudsman code, customer experience (CX) and ASB.
8. Deep-dive into areas that you know impact on resident experience and satisfaction. From KPI audits to evidence-based service reviews Housemark is here for you.
For information about the value of Housemark membership, getting ready for consumer regulation or any of the other topics in this blog – contact info@housemark.co.uk.