Performance testing on its own isn’t useful, it’s a waste of time and money. Testing can be a great method to understanding at a deep level where you can make radical, innovative and disruptive business change. But few use it this way.
Almost every dev/product team out there can at some level of run automated performance testing, it might be very basic but it’s a start. They’ll typically:
- Setup a JMeter or python script
- Run x iterations of a user journey
- See if it was as quick as the last time
- Check-in code
If you are lucky you might get a meaningless report with response times and throughput, maybe even on a graph. But what does good actually look like?
Good companies know what response times ‘should’ be and the relationship to the throughput (users/traffic/something) on the system.
Great companies measure of application efficiency. Monitor and fix stability issues in dev. Consistently instrument their design, using data to influence the future. Create small, flexible, mutable services and applications that scale up, down and across. Forecast and provision the right type, level and placement of capacity, moving with the sun. Build scalable resilience into applications before it’s needed. And of course, know how fast their application should be and how many things it should do at any time.
I’ve talked previously about creating logical maps of systems, end to end. Using business>service>infrastructure data and map this to customer behaviours. Then using this to influence the new products that you bring to market.
Test for insight.
Testing for the sake of testing is wasting everyone’s time. Test for insight. The insight you gain will let you create great products, market to the right audience and stay ahead of the future.
Those who really understand their systems and how they work will have a superior understanding of their customer’s needs and wants. They will be able to make whatever changes they want without worrying about what the underlying tech stack is doing. How liberating would that be?
Testing is often used as a tick box exercise and in most cases, the results you get aren’t even good enough for that. Forget about testing, start thinking about how and where can I get the insight that will let me do something nobody has done before.
Thats where the real value is.
Thomas
You can sign up for more ‘unconsidered needs’ below.
Sign up if you’d like regular insights with ‘Unconsidered Needs’, a short weekly email about the trials and tribulations of tech value creation.