Creating your engagement score is the first step in your Sherlock journey because everything in Sherlock is dependent on the creation of a scoring model.
The beauty of Sherlock is you’re in control (it’s your product after all – you know it best!), but what is the best way to go about creating this model in order to ensure the best results?
This article will outline a few tips and best practices based on some frequently asked questions.
How many events should I have in my engagement scoring model?
You are likely tracking dozens of events in your product, but generally speaking, there are probably only 10-20 that really matter to assessing engagement. We have seen successful scoring models with as little as 5-7 events and some with as many as 20+. That said, 10-20 is a good rule of thumb.
Do be aware that too few events is problematic for creating a good scoring model. If you only use 1, 2, or 3 events, there will likely not be enough diversity in the usage to create a wide enough range to create a meaningful ranking of users/accounts.
How should I weigh each event?
A properly configured engagement score should expose and elevate your “power users.” This means your “power user” features should be given the highest weight (7-10) in your model. These are the features that are not very frequently used, but used by your most advanced and experienced users.
Your most common events – those triggered by every (or the majority) of your users – such as login – should be given the lowest weight (1-3) – or not weighted at all.
Features that fit in the middle – what you might consider your “core” features – should be given medium weights (4-6).
Is there any way I can tell if there I’ve missed any important events that should be added to my scoring model?
One of the most unique features of Sherlock’s Events tab is the ability to identify the events that are driving the most engagement. A few days after creating your first score, you should check this tab – toggle to the “unscored events” and look for engaging events that you might want to add to your scoring profile.
How rigid is this score? Do I only have one shot to make it correctly or can I edit it later without affecting the results?
It is very easy to update and iterate on your Sherlock score. In fact, during your first few days with Sherlock, you will likely be iterating on your score quite a bit. When you change or update your scores in Sherlock, Sherlock quickly refreshes the calculations that drive all your reporting data. You don’t have to worry about your changes destroying your data in any way.
Can I score different user groups differently?
Yes. With Sherlock, you can create multiple scoring profiles.
You shouldn’t worry too much about creating multiple scoring profiles in your first few days with Sherlock, but it is something to keep in mind. Multiple scoring profiles come in very handy when you have different groups of users that use your product differently.
Remember, no one knows your product as well as you do. Because of this, creating your scoring model should be very intuitive. In general, you should trust your instincts here.
