Blog Archives

DEM or YOU? Choose Wisely.

2/10/2019

Dramatic title right? well, it's a pretty dramatic topic. When you choose to not monitor the digital experience, you're sort of slamming the door on your user's face...you're being selfish!

DEM stands for Digital Experience Monitoring or, if you're paying attention, Management. It's the idea of understanding your user experience and how it's perceived by your users. Your site could be running beautifully on a few EC2 instances using almost no CPU or memory but, if the users can't get to what they're looking for, that efficiency is undercut by a bad user experience. That's when you get all sorts of post on forums and social media saying, "Your website is terrible" or, "You need a new IT team".

You can talk to those disgruntled users and try to get screenshots, a customer ID, steps to reproduce (and you probably should) but, this isn't going to help you long term. You probably figured out APM (Application Performance Monitoring) a while back and can at least detect if a server goes down...lets tackle the experience.

BTW, if you didn't watch Community and/or don't know what krumping is, watch this video first...things will just make more sense.

Am I Using DEM?

Chances are, you've probably got some of the tools that could help you better understand your digital experience in place. It could be an analytics tool or a session recording tool but, if you're using these tools in just one department or for just one thing (like measuring conversion), then you're not using it to it's fullest potential. As yourself: "who else would benefit from knowing that 50% of the users that get to this step don't purchase our product?" or, "Would it help my designers to see that users barely interact with this part of the page?"

This is DEM.

A true DEM solution starts with understanding the experience end to end under several different lenses. On the home page, once they've purchased, logging in or out...you have to see it all. Most teams actually struggle with this because depending on what the user is doing, though it's all one site to them, it's actually 3 different apps. A legacy browse app, a newer purchasing app, a totally different account app. Whatever the case though, you need to be able to see it all in the same context across the board.

0 Comments

Synthetic Data vs Real User Data: What Is Each Best For and How Do They Work Together?

2/3/2019

0 Comments

We've had a lot of followers stop by and ask us for help understanding the difference between synthetic user data, and real user data. While we don't have all the answer and specific use cases will depend a bit on your product, I'll break it down into 3 parts to get you started and if you've got more questions after, you know how to find me!

When Is Synthetic Data Useful?

Synthetics are basically automated test scripts that run through your application and perform various test scenarios to determine if you site is working as expected. Because they are pre-recorded bots, they make a few assumptions that would be hard to make when looking at real user data. These assumptions allow you to focus on specific KPIs by removing some white noise. synthetic scenarios are often considered clean room or happy path tests.

Assumptions Made

Your buttons, product info, and overall funnel makes sense - If any part of your website would be confusing to a guest, your synthetic flow would not detect that because you recorded it to complete tasks. It'll go through each page, wait for it to load, and continue where real users might read something that gives them pause or sparks a question the page may not answer.
Your user wants to make a purchase - Synthetic users start browsing your site "knowing" they want to make a purchase so, they usually pick an item, choose a size or color, add the item tho their cart, then buy it. Retention, conversion, and general "shopping" around are not taken into account as a result.
Destructive behaviors don't exist - Scenarios like filtering product results until you get "no products found" are sometimes called destructive behaviors and they happen more often than you think. If a user filters for shoes that come in either red or green, then selects a pair of shoes only sold in white, the search results might come back empty. This causes user confusion and flow abandonment but, it's not detected by a synthetic.

Synthetic users allow you to understand the user experience from a technical side. If my flow makes sense and my content is on point, users will be happy with my site because the page load times are good, content is rich but well-compressed, and key flows have been up and operational all day. Because of that, frequent synthetic runs are a good way to see if key parts of your site are operational without having to wait for real users to try using them.

When is Real User Data Useful?

It's hard to not see the value of real user data upfront and with synthetics explained earlier in this write-up, this should be pretty straight-forward but, let's dive in anyways! Real users are best for understanding patterns and trends but, they can also be used for some real deep-dives into edge cases and even for 1 on 1 customer support.

Example Use Cases

User trends and patterns - Let's say your CDN is experiencing issues in the state of Texas but, you haven't been notified of that yet by your provider and hey, they might not even know. With the right tool in place, a spike in load times from users in Texas or a drop in conversions would quickly throw up a red flag and using real user data and comparing it to historical trends, you've just identified a problem.
Deep-dive analysis - For some reason, people using Google Chrome are 4 times more likely to buy your product than those using Safari...why? Using real user data, you can dive deep and after some hard work, you'll discover that a button that doesn't render correctly on Safari is preventing users from clicking through to the payment screen.
1 on 1 customer support - users call in complaining about your site every now and then. Some might just be looking for a discount but, others are genuinely confused or frustrated. If that user can share their email address with you or maybe a customer ID, you can look for their session and walk through it with them, not only identifying the problem but helping them get what they need.

How Can Synthetics and Real Users Work Together?

Real user data is collected as application users interact with your website and takes into account every click, mouse-over, aimless back and forth...well, it takes everything into account. As a result, it's a lot to sift through or reliably digest without a good baseline. Synthetics establish that baseline for you by measuring the "happy path" for that same scenario. If a synthetic user gets through the flow without issue, the problem is not in the application itself but, maybe in the experience. Is the user confused? Is the right information being given? Is the price too high?

0 Comments

DEM or YOU? Choose Wisely.

Am I Using DEM?

This is DEM.

Synthetic Data vs Real User Data: What Is Each Best For and How Do They Work Together?

When Is Synthetic Data Useful?

When is Real User Data Useful?

How Can Synthetics and Real Users Work Together?

Author

Archives