One of the most overlooked aspects of DevOps software development is the pursuit of meaningful metrics that provide accurate insights into the customer experience. It may come as no surprise, but the process is far from easy. Measuring and validating a user story requires an ability to adapt the approach to gathering metrics, as well as consistent action through every iteration, as well as an organizational commitment to improving CX.
Developers need to understand precisely how certain features or bugs are affecting end-users. Business analysts and product managers want to build connections between user experiences and business objectives. Operations and QA teams need insights into how their work is affecting UX so they can more effectively support the broader development lifecycle.
Despite the challenges, it’s in an organization’s best interests to measure and adjust the user experience throughout the entire development lifecycle, a process that requires a myriad of tools and methods to pull off successfully. What follows are a few techniques that I’ve found have worked.
Inspecting and Adapting Through DevOps Practices
Alongside the ability to deploy fast and deploy frequently, DevOps practices allow companies to acquire more feedback from users in a shorter time span. However, if companies deploy software several times a day without measuring and responding to user feedback, it can become more difficult to respond to or deal with negative impacts on customer experience.
At its core, Agile is not only about maximizing development velocity, but also about inspecting and adapting, which is why DevOps — as an extension of Agile — focuses on enhancing the customer experience through the same approach. The difference is that the DevOps approach leverages operations, thus using telemetry and monitoring tools that make inspection and adaptation much easier.
Enhancing CX is all about devising metrics, measuring them, adding to them, and then adjusting them if they don’t provide a complete perspective, which aligns extremely well with the iterative and versatile DevOps methodology.
Related Article: Does Your Organization Really 'Get' Agile?
Before an organization can measure user experience, the company’s decision-makers must be able to effectively communicate business objectives to developers and engineers. Without this knowledge, front-line teams cannot implement the tools or mechanisms required for a meaningful understanding of CX.
From product management up to organizational leadership, the people who are best trained to think like users should be suggesting the most valuable metrics that are well-aligned with the user experience. Armed with that guidance, developers can then dive into the technical aspects of delivering those metrics and validating them in a prototype.
It’s this level of collaboration that is core to the DevOps ethos and results in the best measurement of user experience while also helping organizations reach business goals at maximum velocity.
Related Article: Should We Hand Over UX to Just Anyone?
Empathizing With Users
End-users are not generally in a position to judge your infrastructure specifications or coding practices, but they will be quick to judge the experience that the application or service provides, especially when that experience is a negative one. It’s vital to put yourselves in the user’s shoes to experience a different mindset than the internal, organizational perspective of the application.
You can achieve this by identifying specific points in the user journey that have a direct impact on user experience. For example, with a ridesharing app like Uber, there is a specific workflow — book a car, take a journey, and auto-process the payment — but imagine they added a new feature that requires users to negotiate fares with drivers. Uber would need to track how many people are engaging in the new process, whether or not they are drawing it out for too long, or if they are ignoring it altogether.
By analyzing specific behaviors during the iteration of new features you can determine whether the additions are too complicated and how users might respond to them, building an empathetic understanding of the user story. This is also sometimes referred to as product-oriented monitoring, which allows you to monitor the impacts or usage rates of new features and quickly understand how well-accepted or useful they are.
Related Article: Forget These 3 Myths About DevOps Feedback Loops
Granular Actions That Converge Into Great UX
While there is no one-size-fits-all technique for measuring how users interact with an application, there are plenty more options worth exploring.
For instance, you could mine the application’s logs or include a feature that records usage data and relays it back to the company for analysis, with user approval (think Apple or Google’s user tracking agreements). While you should certainly provide users with the ability to send direct feedback, parsing that feedback can be costly when dealing with thousands of emails or social media complaints a day, particularly when the feedback varies significantly.
Striking a balance between good feedback and potentially expensive feedback is also key. In some cases where a single user is adamant about a specific change, while it might be an obvious improvement to the application, it might also be a costly feature to implement, so unless more users make the same request, it wouldn’t make much business sense to spend time and resources on the fix.
User interviews are another popular approach as a simple means of gathering feedback, but they might not be as meaningful as metrics and data. There will always be a certain level of bias to be aware of, especially in large user groups, but common ground is easier to find when interviewing smaller user bases, so one organization might get more value from these sessions than another.
In the future, we expect to see a significant evolution in the use of big data techniques to allow organizations to better assess unstructured data collected through various user feedback channels. Until then, be sure to assess how your organization is gathering metrics, measuring them, and actively adapting to maximize user experience during the DevOps development lifecycle.
Related Article: Not All Customer Feedback Requires Change