by John Thompson
As the world of ecommerce continues its relentless advance, ever more sophisticated technologies are being developed to increase conversion. Personalisation, optimisation, analytics and multivariate testing technologies are receiving significant investment from both vendors and ecommerce sites, and with good reason since a healthy return on investment can be achieved.
These technologies are the icing on the ecommerce cake, but the relentless focus on them can obscure a big problem: the cake itself may be crawling with bugs.
Bugs can seriously damage your site’s health
For good commercial reasons ecommerce as a discipline is usually controlled by marketing people rather than by technologists. These professionals are very motivated by marketing technologies, but sometimes they underestimate the importance of the deeply unsexy problem of cleaning up bugs, which are especially prevalent after a site upgrade.
It has happened to all of us. On Mothers’ Day this year I was undertaking my usual last minute online purchase of flowers (on a leading site). After pressing the order button there was a long delay and I was then returned to the order page again, with no idea whether or not I had ordered. Days later I was booking a ski lesson for my kids and there was a catastrophic website melt down with software code all over my screen. I have worked with ecommerce customers whose otherwise commercially successful websites generated thousands of server errors every day. This is a really prevalent problem.
When bugs occur it can be catastrophic to the customer experience. An error can cause a fundamental breach of trust with a customer, especially at that most vulnerable moment when a credit card has just been retrieved from a wallet.
What’s the problem?
So why do we often agonise over A/B testing of the aesthetics of the new user interface, while neglecting these more fundamental technical problems?
The heart of the issue is replication. Glitches that happen to all customers, all of the time, are fixed quickly. Demand drops, emergency meetings happen, there is shouting and screaming and rapid resolution.
However, most bugs are subtle insidious creatures and are not easy to find. Bugs may be triggered by any combination of prior steps in a user’s journey or the product being purchased, browser type, client operating system, network quality, customer’s country of origin and language, site volumes and other factors. The combinations run into billions and it is not possible to test exhaustively prior to release, and so in many cases it is the customer who finds the bug, and not the tester.
When the glitch occurs, damage is done, and the very same bug often goes on to undermine the customer experience again and again. Bugs are not created equal, but the worst ones on the biggest sites can cost millions in lost revenue. Sometimes these baddies are busy causing their subtle havoc while their very existence is doubted. I know of a number of ecommerce vendors where customer services passionately believes there is a bug in the conversion journey, but IT, having checked the log files, insists the problem is entirely mythical.
How do bugs survive in the wild?
How can the very existence of a reported bug be a subject of debate in the 21st century? Firstly, when a problem occurs customers generally don’t report it. One of our retail customers recently estimated as few as one in 1000 customers call when they experience a bug. I can believe this, I never do, I just move on to another vendor.
When that highly valuable inbound phone call does occur, it often goes badly. The customer is not a software tester; they just want some flowers (or maybe a ski lesson). The customer is not going to be able to describe all the steps they took leading up to the error. They are unlikely to be able to give chapter and verse on their own technical set up, or even the exact time the error occurred.
Equally, the customer services agent is not a software tester, and probably won’t be able to capture the necessary details, even if they are lucky enough to be speaking to an unusually technical customer. This means the IT team will, at best, get a garbled message. It is unlikely they will get enough from this, even combined with the log files, to find the problem. They will often just put the report down to user error, and deny the bug exists.
It doesn’t have to be this way
It is possible to turn every customer into a software tester who records their every step, so that every error (and the lead up to it) can be replayed both visually and from a technical perspective.
This is achieved using a technology known as Session Replay, and there are a number of good vendors in the space. With session replay implemented, when an error occurs this can be flagged up directly, avoiding the wait for a customer to call.
It is then possible to analyse exactly how often the error is occurring and hence its cost in lost conversions. It is also possible to understand exactly what caused the error, and hence it can be quickly replicated and fixed. A number of our retail customers have reported an order of magnitude, or better, improvement in the efficiency of bug fixing using this kind of technology. There are no more mythical bugs, they are real and are found and fixed.
Econsultancy’s 2012 report on Reducing Customer Struggle suggests that only 17% of e-commerce vendors use session replay, yet 57% recognise it as very effective. So why is there this gap between belief and practice?
Firstly the technology is not as sexy as the marketing-led tools, and the RoI, while in reality compelling, is not always obvious to a marketing person who is likely to underestimate the prevalence of bugs that impact conversion. Additionally, the most appropriate solutions for bug fixing require on-premise installation, not SaaS, so implementation requires more planning. While SaaS is easy, and trendy, on-premise still has its place for some solutions. In this case on-premise systems can capture every single client-server interaction, and hence all the problematic ones.
Econsultancy’s findings suggest that the time for session replay is nigh. Just as a plane would never fly without a black box, in the future websites will have their own black-boxes which will record all web traffic and ensure that when crashes do occur, they can be properly investigated and avoided in the future. These website black boxes will ensure that customer experience improves, and that conversion-sapping bugs are consigned to history so that your site will always be fighting fit.
John Thompson is CEO of website session replay specialist, User Replay.
You are in: Home » Guest Comment » GUEST COMMENT Is your site fighting fit?
GUEST COMMENT Is your site fighting fit?
by Staff Writer
This is an archived article - we have removed images and other assets but have left the text unchanged for your reference
by John Thompson
As the world of ecommerce continues its relentless advance, ever more sophisticated technologies are being developed to increase conversion. Personalisation, optimisation, analytics and multivariate testing technologies are receiving significant investment from both vendors and ecommerce sites, and with good reason since a healthy return on investment can be achieved.
These technologies are the icing on the ecommerce cake, but the relentless focus on them can obscure a big problem: the cake itself may be crawling with bugs.
Bugs can seriously damage your site’s health
For good commercial reasons ecommerce as a discipline is usually controlled by marketing people rather than by technologists. These professionals are very motivated by marketing technologies, but sometimes they underestimate the importance of the deeply unsexy problem of cleaning up bugs, which are especially prevalent after a site upgrade.
It has happened to all of us. On Mothers’ Day this year I was undertaking my usual last minute online purchase of flowers (on a leading site). After pressing the order button there was a long delay and I was then returned to the order page again, with no idea whether or not I had ordered. Days later I was booking a ski lesson for my kids and there was a catastrophic website melt down with software code all over my screen. I have worked with ecommerce customers whose otherwise commercially successful websites generated thousands of server errors every day. This is a really prevalent problem.
When bugs occur it can be catastrophic to the customer experience. An error can cause a fundamental breach of trust with a customer, especially at that most vulnerable moment when a credit card has just been retrieved from a wallet.
What’s the problem?
So why do we often agonise over A/B testing of the aesthetics of the new user interface, while neglecting these more fundamental technical problems?
The heart of the issue is replication. Glitches that happen to all customers, all of the time, are fixed quickly. Demand drops, emergency meetings happen, there is shouting and screaming and rapid resolution.
However, most bugs are subtle insidious creatures and are not easy to find. Bugs may be triggered by any combination of prior steps in a user’s journey or the product being purchased, browser type, client operating system, network quality, customer’s country of origin and language, site volumes and other factors. The combinations run into billions and it is not possible to test exhaustively prior to release, and so in many cases it is the customer who finds the bug, and not the tester.
When the glitch occurs, damage is done, and the very same bug often goes on to undermine the customer experience again and again. Bugs are not created equal, but the worst ones on the biggest sites can cost millions in lost revenue. Sometimes these baddies are busy causing their subtle havoc while their very existence is doubted. I know of a number of ecommerce vendors where customer services passionately believes there is a bug in the conversion journey, but IT, having checked the log files, insists the problem is entirely mythical.
How do bugs survive in the wild?
How can the very existence of a reported bug be a subject of debate in the 21st century? Firstly, when a problem occurs customers generally don’t report it. One of our retail customers recently estimated as few as one in 1000 customers call when they experience a bug. I can believe this, I never do, I just move on to another vendor.
When that highly valuable inbound phone call does occur, it often goes badly. The customer is not a software tester; they just want some flowers (or maybe a ski lesson). The customer is not going to be able to describe all the steps they took leading up to the error. They are unlikely to be able to give chapter and verse on their own technical set up, or even the exact time the error occurred.
Equally, the customer services agent is not a software tester, and probably won’t be able to capture the necessary details, even if they are lucky enough to be speaking to an unusually technical customer. This means the IT team will, at best, get a garbled message. It is unlikely they will get enough from this, even combined with the log files, to find the problem. They will often just put the report down to user error, and deny the bug exists.
It doesn’t have to be this way
It is possible to turn every customer into a software tester who records their every step, so that every error (and the lead up to it) can be replayed both visually and from a technical perspective.
This is achieved using a technology known as Session Replay, and there are a number of good vendors in the space. With session replay implemented, when an error occurs this can be flagged up directly, avoiding the wait for a customer to call.
It is then possible to analyse exactly how often the error is occurring and hence its cost in lost conversions. It is also possible to understand exactly what caused the error, and hence it can be quickly replicated and fixed. A number of our retail customers have reported an order of magnitude, or better, improvement in the efficiency of bug fixing using this kind of technology. There are no more mythical bugs, they are real and are found and fixed.
Econsultancy’s 2012 report on Reducing Customer Struggle suggests that only 17% of e-commerce vendors use session replay, yet 57% recognise it as very effective. So why is there this gap between belief and practice?
Firstly the technology is not as sexy as the marketing-led tools, and the RoI, while in reality compelling, is not always obvious to a marketing person who is likely to underestimate the prevalence of bugs that impact conversion. Additionally, the most appropriate solutions for bug fixing require on-premise installation, not SaaS, so implementation requires more planning. While SaaS is easy, and trendy, on-premise still has its place for some solutions. In this case on-premise systems can capture every single client-server interaction, and hence all the problematic ones.
Econsultancy’s findings suggest that the time for session replay is nigh. Just as a plane would never fly without a black box, in the future websites will have their own black-boxes which will record all web traffic and ensure that when crashes do occur, they can be properly investigated and avoided in the future. These website black boxes will ensure that customer experience improves, and that conversion-sapping bugs are consigned to history so that your site will always be fighting fit.
John Thompson is CEO of website session replay specialist, User Replay.
Read More
You may also like
Register for Newsletter
Receive 3 newsletters per week
Gain access to all Top500 research
Personalise your experience on IR.net