The term “User Engagement” has been bandied around the IT industry for decades now. To my knowledge the only other industry that refers to its customers as “Users” is the illegal drug industry. There are some similarities. The users are demanding, can’t get enough of what they want, pay too much and inevitably end up disappointed.
Jokes aside, however, when it comes to systems designed to facilitate collaboration, like Enterprise Social Networking (ESN) software, it leads to confusion as to what “user engagement” really means. IT heritage thinking would have it mean “how the user engages with the IT system”. Its all about how users enter something into a system and get something back out in return. So when it comes to measuring and monitoring how successful an IT system is, it invariably comes down to usage. How much was the system exercised? How available was it? Did the user get out of it what they wanted?
When we talk about collaboration and collaboration systems, our heritage IT thinking kicks in and we want to measure how many people are using the system as the key measure of success. So we start to measure User Engagement in terms of system utilization. But what about how users engage with each other? Isn’t that what collaboration is all about? Isn’t that the real ultimate measure of user engagement? It’s not that system utilization is irrelevant, its not. But if the focus is on the system utilization we could lose sight of what the real objective is. Think of it like a dating site that measures its success by how many people they sign up or how many profile searches are made, rather than measuring themselves through the true measure of successful matches. It’s unlikely that such a site would have a very long life.
At SWOOP we focus on relationship centered metrics. We believe that user engagement should be about how people engage with other people in collaborative activities. As part of our ESN comparative analytics framework, we include the following “User Engagement” measures:
| Measure | Calculation | Rationale |
|---|---|---|
| Mean 2-Way connections | A two-way connection exists when an activity is reciprocated e.g. You ‘liked’ my post and I ‘replied’ to your post, or similar. | Looks to mimic reciprocity. The higher the score, the more ‘engaged’ users are with each other. Reciprocity is a foundation element of trust, which underpins effective collaboration. |
| Post/Reply Ratio | Total number of posts divided by total number of replies. | Seen as ‘pump priming’ for the network. A value > 1 may be required to launch a network but should settle below 1 once traction is achieved. See our persona post {Ref} |
| Recognition | % of Likes and Mentions | A ‘Like’ or a ‘Mention’ is a form of recognition, something that should be encouraged in a social system. Recognition promotes user engagement. |
| Direction | % Notifications | A ‘Notification’ is like a ‘cc’ in an email. Usually it is used to direct attention of the receiver. A common management activity. |
| Response Rate | % of discussion threads longer than a single post | In the absence of context, this is the best measure of ‘value’ generated. A high response rate shows that people are benefiting in some way from participation. |
| %Public | % of messages that are public messages, is posted in a group (public or private) or to all. (Private messages are those that are sent directly between users using the ‘Send message’ feature. It is the same as email). |
The objective of an enterprise social network is to promote transparency. Measuring the split between public and private message tells us the extent to which transparency is being achieved. Users are more likely to freely collaborate in open and transparent environments. |
We suspect that the above measures might be quite foreign to those used to the traditional systems utilization measures. But just like the dating site, until we start to measure success in human terms, rather than system activity measures, our users will continue to pay too much and still end up disappointed.