, ,

Building a Product from an Idea : The Lean Startup Way

We all have ideas. We all have felt the need of having something more to an existing solution or an alternate way of doing something which we often do. Startups and products are born out of this. The good part or maybe the sad part is that there are thousands of such ideas and products that spring up every day and it becomes increasingly difficult for these products to succeed in the market.

Of course having a great founding team with the right mix of technical, marketing and design skill set would go a long way in helping the product to wade through the clutter and be noticed but it still doesn’t guarantee the success of the product. It’s often easy for a founding team to lose direction early on in terms of what’s the right product that people are willing to use, or better, willing to pay for. There is an even worse scenario which I have often seen among founders when they try and convince themselves that the features and the products that they are building is the right solution based on intuition and practically zero metrics to back their claim. That’s suicidal.

It’s imperative for a startup to follow the Lean methodology’s Build-Measure-Learn loop. But before you enter the build phase, your first step should always be to do Research and understand the market you are going to target.

Research

First things first. One wouldn’t want to waste a significant amount of resource on an idea which has relatively zero market potential. So always begin by understanding the true market potential of your idea. You can start by asking yourself a set of questions initially:

  • Will my idea address a genuine pain point, if yes, what is it?
  • Who will be my potential customers and where can I find them?
  • Who are my competitions?
  • How different is my idea from what my competitions have?
  • Will I pay for a product like this? Would anyone pay for the product I intend to develop?
  • Are there are regulatory constraints?
  • What would my rough budget be and what would be the resources required for a basic product?

I’m sure you won’t get comprehensive answers to a lot of these questions but then the point of asking yourself all these questions initially is it helps you understand the market and the opportunity you are going after and sets the context right. Googling will give you sufficient inputs which will enable you to take a call on if it’s worth pursuing further. If you want to understand things a little deeper, do a survey or shortlist a set of people who will in the future be interested in the product and try and get their opinion on if they would actually pay for such a product (To be honest, at this stage it’s difficult to really understand if the users will pay for it at this stage, but do get opinion from people nevertheless.)

In already established markets there would be a fair number of research reports which you can leverage to understand in detail the market you are going after. An easier way would be to use Google Keyword Tool or Market Samurai to understand the demand for your idea. It’s always easier for a startup to build something in a space where there is an existing demand and is not fully saturated than to carve out an entirely new market. I am not saying that’s not possible but with limited resource at your disposal in your early days, trying to create a new market might not be the best option.

image

Just did a search on Google KWT for the term Video Games and see the results. It has a fairly good number of searches worldwide. The KWT also gives you a set of related keywords which might help you even segment the entire market.

It’s important at this stage to try and segment the market you are going after. Having a generic solution won’t help at an early stage. Segmenting the market you are going after gives you a much better chance of validating your idea. The Idea can be expanded on to other segments as and when you grow and become mature. Also, make a shortlist of your competition and their offerings. This would give you a fair bit of understanding on the current market demand for various features and would allow you to understand how your product is different from your competition.

All of this helps you in getting a Problem/ Solution fit. It’s good to get a feel of the market even before you start prototyping and building a product. Like Eric Ries mentions in his Lean Startup methodology, it might be a good idea to just create a landing page and put up a “Register to get an Invite Option” and check how many click through to register and actually register. This is a trend followed by a lot of online startups and especially Apps. One of the most important tactics for an app’s pre-launch marketing strategy is to build up a landing page with an option for the users to subscribe to be notified when the app goes live. This would also enable you to get a feel of the solution you are suggesting for a problem. Again, the problem here is that often people without proper segmentation and without trying to get their target users to come on to the page would conclude that the idea has no demand in the market. This is why it’s important to segment your market and know your core group of audience. Hunt for them on forums, groups or anywhere they are available if you want to make people discover your webpage for free or else use Google Ad words or any of the Ad solutions to target your core group of audience. Understand if there is a demand for your solution.

The next stage in the product lifecycle is to develop an MVP (Minimum Viable Product) that would actually enable you to reach out to customers, engage with them and understand better the demand for the product.

Minimum Viable Product

The concept of a Minimum Viable Product was introduced by Eric Ries, the man behind the Lean Startup movement. In his own words :

“The idea of minimum viable product is useful because you can basically say: our vision is to build a product that solves this core problem for customers and we think that for the people who are early adopters for this kind of solution, they will be the most forgiving. And they will fill in their minds the features that aren’t quite there if we give them the core, tent-pole features that point the direction of where we’re trying to go.

So, the minimum viable product is that product which has just those features (and no more) that allows you to ship a product that resonates with early adopters; some of whom will pay you money or give you feedback.”

According to me, it’s always a difficult task clearly understanding what exactly is “minimum viable” as far as your product/ idea is concerned. It would be different for each idea and category. Understand that if the product is as is any other competitor and there is no differentiation then the product you are shipping is in no way a “minimum viable” product. Focus on your core value proposition and how your product is different from the rest. If your differentiation is purely the experience that you give your users then ensure that when you ship out your MVP, you enable your customers to have that experience. Minimum Viable Product does not mean that you roll out a crappy product. In fact that would be suicidal as with Social Media these days it does not take a lot of time to completely kill your product or brand with a negative word of mouth. Of course the MVP can have bugs and there would be hundreds of features that could be added later. The early adopters that you manage to get are always going to give you a leeway and that’s because they genuinely need and value the core experience or the core feature your product provides. So ensure that the core proposition is in its entirety is reflected in the MVP.

Steve Blank in his book outlines the four stages to the Customer Development process with the following success end goals:

  1. Customer Discovery – Achieve Problem/Solution Fit
  2. Customer Validation – Achieve Product/Market Fit
  3. Customer Creation – Drive Demand
  4. Company Building – Scale the Company

This is a great framework for someone operating with the Lean Startup methodology. The initial research phase and the development of the MVP falls under the first bucket where in one achieves the Problem/ Solution fit. This does involve effort however you do significantly cut down on the unnecessary resource you would have spent otherwise on trying to create something which has no demand in the market only to realize that after you have pumped in all of your money and effort.

The Second phase of Customer Validation is where one achieves Product Market fit. This is the stage where in you actually try and sell your MVP and or make your customers to use it to tweak and bridge the gap between the Product and the Market.

Achieving Product/ Market Fit:

How exactly does one determine whether you have achieved product/ market fit? Different people will give you different definitions for Product/ market fit

“Product/market fit means being in a good market with a product that can satisfy that market,” according to Marc Andreessen

Andrew Chen

image

image

Sean Ellis has created another metric for determining Product/ Market fit. He suggests asking existing users of a product how they would feel if they could no longer use the product. According to him, achieving product/market fit requires at least 40% of users saying they would be “very disappointed” without your product.

For me the whole idea of getting a Product Market fit is nothing but getting to a point with your product when a particular segment of the market which you have identified as your initial target segment embraces your product so that you can grow your company/ product scalably. Achieving Product/ Market fit as early as possible is crucial for any product as it allows you to then focus on company growth and not on iterating and pivoting the product. Spending significant money and effort on growth and marketing at this stage before product/ market fit is not an advisable strategy.

It’s important for startups to constantly measure during this stage and understand the behavior of their users. One needs to craft and test several value propositions, user flows, conversions, user interactions to effectively achieve a product/ market fit.

The priority here is to focus on the macro metrics, the right ones. Understand that optimization of micro-metrics comes at a later stage once we achieve product/ market fit. There are various macro metrics that matter; you may refer to Dave McClure’s AARRR model.

  • Acquisition – How many people landed on your website coming from a marketing campaign or through viral channels that you are tracking and then you acquire the user.
  • Activation – The user uses your product and completes a core action on the platform.
  • Retention – What is your churn? How many of the users you have in your user base are active? How many stopped being active and why?
  • Referral -How many of the users that are using your product are willing to refer to others?
  • Revenue -How many users are willing to pay you of the ones that are using the service?

During this stage out of the 5 macro metrics Dave suggests, there are only two that needs to be tracked comprehensively. They are: Activation and Retention.  Of course Acquisition is important as well because for measuring and optimizing activation and retention there needs to be sufficient users. But then the idea here is to not spend and focus on acquisition but to focus on Activation and retention in a core segment by minimizing your acquisition cost and optimizing it. Try and figure out the best and most effective channels to let your target audience discover your product and allocate a budget accordingly. Social Media these days provide a great channel for enabling your target users to discover your product, so utilize it to the maximum effect possible.

Try and map out the important actions on the platform that corresponds to the macro metrics : Activation and Retention.

Activation:

Activation rate, in a nutshell, is the percentage of users who stick with your app long enough to experience the value it offers.

For a project management application, the point of first value might be when an account achieves these things:

  1. Account created
  2. Invited 2+ team members
  3. Created project
  4. Uploaded 2+ files
  5. Created 3+ calendar events
  6. Created 1+ tasks
  7. Completed 1+ tasks

And these actions can be taken in any order – a non-linear experience – so measuring them as a funnel is a mistake.

In this case, you should be measuring Activation as a percentage – not a linear funnel. You need to measure, how many of these steps have been completed (regardless of the order). If a new user or account needs to complete 5 steps to become fully activated and has only completed 2 of those steps – they are 20% Activated. If it completes 4 of 5 steps – they are 80% Activated.

This view of number of criteria for activation fulfilled by the end user gives a clear picture of how well New accounts are able to complete the list of activities you have defined as activation criteria.

Retention:

Retention is nothing but getting the users back on the site regardless of the engagement they have on the site. You can define retention as mentioned or tie it to certain key actions on the platform. In general for a consumer product which is both creation and consumption based, it might be good to just consider the activity of the user coming back to the site as retention. For eg: Facebook or twitter might consider retention as the case of users just logging back in to the site. Engagement, however, is a different concept where a platform like Facebook or twitter would want the user to perform any major/ core user action on the platform like sharing content, liking or updating status or tweeting etc.

A good retention rate would be different for different consumer products/ apps depending on the nature. It would also depend on the customer usage cycle which tends to be shorter for a social gaming app while it tends to be a little longer for a platform like Snap, TikTok etc. So based on your product’s customer usage cycle and general trend in your niche/category decide on your target retention number/ time frame ( 1 Day, 7 days or 28 days) to achieve.

Measure and iterate on both these macro-metric to get to Product/ Market fit. Use Funnel and Cohort analyses to better understand the user flows and the churn at each stage so that you can identify and improve/ rectify the non-required or wrongly crafted features and flows. Breakdown each user flow to understand in depth any issue there is. The idea here is not optimization for efficiency but the idea here is to validate your MVP. People often relate A/B testing with changing colors of the Sign Up button, yes, that might be a good way to improve on the conversions in some cases, but getting to product Market fit is all about validating your MVP, to get people to buy into the features or the experience it provides and then make them repeatedly come back to the platform. There would various broad scenarios:

Have high arrivals but low Conversions: Tweak your messaging and positioning to check if that helps in conversion. Also, ensure that the incoming traffic is composed of people you assume to be your target audience.

Have low arrivals but high conversions: Work on the channels to bring in more targeted traffic. Groups, Forums, Meetups etc of target community would be a great start. Try and improve on the keywords you chose for your PPC campaigns.

Have high conversions but low activation: Ensure people understand the interactions on the platform. Is it too difficult to understand or complete the core action on the platform? Would an interactive guide in the beginning help the user understand user actions on the platform?

Have low conversions but high activation: Are you bringing in the right traffic on the platform? Is the messaging right on the front page? Is the signup process easy enough or have you made it too difficult? Is there a clear call to action on your landing page?

Have low activation but high retention: A good sign to have a high retention number. However lower activation would mean either people are not interested in the core activity you have considered or people are not given an easy enough option to complete the core activity on the platform.

Have high activation but low retention: Low retention could be due to lack of interest in the product and it’s core feature. A product which genuinely solves a problem for a sect of people would have high retention numbers. Products which are not a must but is a luxury like Quora would need to constantly remind people and get through the clutter to improve on their retention numbers. Work on either.

The whole cycle would look something like the figure below. Keep measuring all the important metrics, learn and iterate on important features/ flows till you get to product/ Market fit. The earlier, the better.

image

Post achieving Product/ Market fit, the company can focus on user growth and leverage their marketing spend to speed up the entire process. Utilize the best and the most effective channels to scale further. You can now utilize Funnel analytics and Cohort analysis to measure each of the important macro-metric defined in the framework above.

, ,

Customer Engagement – What are the key metrics to track and why?

B2B SaaS is extremely competitive especially for horizontal SaaS products. If you are in the SMB space then that makes it even more challenging for you to survive and then grow. There are a few important metrics the product needs to track assiduously –

  • CAC ( Customer Acquisition Cost)
  • LTV ( Customer Lifetime Value)
  • Payback Period
  • Churn
  • NPS ( Net promoter Score)
  • Sales Velocity

I’m sure most SaaS companies do track these numbers. The key to success is to reduce Churn, CAC and to increase LTV, NPS. One of the key factors that enable a SaaS product to achieve this is customer engagement. But how do you define and measure customer engagement?

 

What is Customer Engagement?

Customer engagement is the interaction/ activity of your customer on the platform. The customer engagement could be a positive or a negative one and it’s equally important to understand the nature of this engagement.

  • A negative engagement increases the risk of Churn, so there are immediate actions that need to be taken to ensure the customer stays.
  • Similarly, a happy and engaged customer provides you with an opportunity to up-sell or cross-sell.

 

So, how do you measure Customer Engagement?

Measuring customer engagement inside the product is the same process as lead scoring at the top of funnel. I had covered lead scoring earlier. Lead scoring is a top of the funnel score that we use to qualify leads based on their activity or interaction with various assets/ touchpoints of the product. You could measure customer engagement with either of the two options:

(1) Use 3rd part software tools that let you define and analyse various events inside the product. Here are a few tools you could consider

(2) Setup your own system where you log various datapoints in your DB and run queries to analyse the same.

In either case, you would have define the important events of engagement and also assign points for these events which would help you calculate the all important engagement score. The events that need to be tracked would be based on the application. For eg:

Helpdesk Software: Add support email, setup forwarding rules, setup DNS, Added Agent

A/B Test SaaS App: Create Test, Start Test, End Test, Share Results

Online Billing APP: Create Invoice, Send Invoice, Receive Payment

Once you have defined the events you can log them and also assign weights to each of these events to calculate your Customer engagement score.

Customer Engagement Score = (wt1*e1) + (wt2 * e2) + … + (wt# + e#)

where wt is the weight assigned and e represents the event being tracked.

Along with the consolidated user engagement score, you could also monitor certain specific or low level metrics that again define user engagement. A few examples are:

  • Daily Active Users ( DAU)
  • Weekly Active Users ( WAU)
  • Monthly Active users ( MAU)
  • DAU/ MAU Ratio
  • User Retention – Day1, Day7, Day30

The core metric that you need to track varies from product to product/ app to app. It’s for you to decide what numbers matter for your product.

 

What next?

Capturing and understanding these metrics defined above is the first step. Setting up steps to improve on these metrics is the next step. This entire process can be automated using a comprehensive automation tool like Marketo, Autopilot, Hubspot Enterprise etc. The right set of messages at the right time goes a long way in optimizing each of the above metrics.

An example:

Pipefy is a great tool for workflow/ process management. It lets you organize all your processes in one place. On signup up with Pipefy, they send you a set of emails to increase engagement.

One of the first emails that they send is a library of pre-existing templates ( most used ones) which would enable the users to get started immediately.

customer-engagement-1

They track weekly retention and send out a mailer to engage the inactive users. This is the second email they send out to inactive users –

customer-engagement-2

Then they follow it up with this email within a few days:

customer-engagement-3

Another example is how Groove improved customer activation using customer engagement data. Grove is a helpdesk software and one of the first things that a user should do after signing up is to setup a support email. They also measure the avg. time it takes for the user to setup the initial support email and if that doesn’t happen then they send an automated email. Here’s the template they use :

customer-engagement-4

They also track user retention and sends out mailers to inactive users to re-engage them. Here’s the template they use for that.

customer-engagement-5

These are proactive measures you can take to increase engagement and user engagement. You can personalize these messages/ automated communications that go out further by segmenting the data. An eg: For Horizontal SaaS products you get registrations from a bunch of industry verticals. You can further segment the user data based on industry vertical and send relevant use case for the industry/ use terminologies that the prospect could relate to. At FieldEZ, we segment prospects based on Industry and the use cases differ across Industry. FieldEZ is used as a Lead/ Sales management tool in industries such as BFSI, Pharma while it’s primarily used for Ticket Management in the Consumer Durables or Manufacturing industry segments.

Other than this customer segmentation also helps in:

  • Identifying what features matter most to a particular segment
  • Measure LTV, CAC, Payback Period, Churn, NPS etc for each segment and work on optimizing the same
  • Measure profitability of each segment
  • Test separate user onboarding techniques for each segment – Messaging and Core interactions based on what matters to the segment
,

Using Data & Analytic Tools to Better Understand Your Users – Measuring the Right Metrics

Products function in an extremely competitive landscape vying for every impression it can get among the millions of potential customers available online. Getting your startup visible or discoverable is one thing, getting them to convert on your website and retain them is an even tougher task with the plethora of services and products that the consumer is forced upon. This is why it becomes so very important for products to understand each and every activity of the user right from the first time a potential customer/ user discovers their service or product on the web to the point they convert and start coming back to their website.

There are plenty of data that’s available to internet products these days and a vast variety of analytic tools to analyze them as well. A few years back, one would have managed analytics and data tracking using just a Visitor analytics tool like Google Analytics, but that is no more the case now. With growing competition, you have far less room to fail. Based on your website and your requirements you can choose from the various Analytic Tools that’s available to you. More often than not, you would need to have a combination of these tools below to better understand user behavior. The below chart gives you the various classes of Analytic tools and their strength in measuring various parameters:

image

Source: www.moz.com

It is crucial for a marketer to appreciate the insights data can provide on user behavior and take necessary actions to correct and optimize wherever required. It is also crucial for a marketer to measure the right data and understand it’s essence for better improvement of the customer lifecycle on their website.

In my previous post, we had discussed the importance of measuring the right macro metrics. For understanding and validating Product/ Market fit, one needs to measure Activation and Retention. However to completely understand the lifecycle of the Customer one needs to also measure the other three elements: Acquisition, Revenue and Referral.

image

Funnels are a great way to understand user behavior on your website. They are visual, simple and map well to most of the events related to measuring the macro metrics. But Funnels alone have their limitations as well. Imagine if you wanted to measure the impact of repeated product iterations you have pushed out to during a period on the revenue. It becomes extremely difficult to track the same using only funnel, one because the impact on revenue is a long term thing and also because you would need to segment users who signed up during the period when each iteration was rolled out to effectively understand the impact on revenue for the set of users who started off with a particular variation of the product. This is where cohorts play an important part. Think, I would cover cohorts in the next post and explain in detail the methodology to track metrics like retention, revenue, impact of feature iterations on both and more. In this post, we will focus on using Google Analytics in tracking the channels resulting in any of your user interacting with your brand, converting on your product/ service and also on coming back to your product/ service. The Digital Marketing Funnel as represented in the figure earlier can be broken down in to 3 components:

  • TOF – Top of Funnel
  • MOF – Middle of Funnel
  • BOF – Bottom of Funnel

Top of Funnel:

Top of the funnel represents the first interaction a user has with your brand/ product. There are plenty of channels on which the interaction would happen and one would need to optimize for each of the channels the interaction happens on. The best solution is to always focus on at max two of the channels where the interactions seem to be most effective. With the new Universal Google Analytics Tool, you can get the channel details at Acquisition » All Traffic.

image

The above table gives you a good understanding of all the various channels that drive traffic on to your platform. You can export the data to an excel sheet and then use a pivot table to understand what medium acts as the best option to drive first time traffic so that you can focus and optimize for that channel/ medium.

You can drill down further to understand the best referral sources through Acquisition » All Referrals

image

Determining which sites have referred the best traffic to your website is important as it enables you to focus on those channels. You can focus on important parameters like Bounce Rate and Time Spent on site to understand the engagement of the users coming from various channels. Not only that, you can also identify websites that are similar to the ones driving traffic on to your website by doing a search on Google [ Use the search query related:”site name”]or on Similar Web to try and leverage on to the similar audience on those sites to generate traffic. For eg: If weheartit.com is a major referrer to your site, then doing a search for related websites on google gives you these results:

image

The above search result gives you a healthy number of similar sites with similar target audience who would be interested in your site. Refining and cross-posting your contents across these websites can also help you in getting additional traffic. You can even automate a few of these by using a service like IFTTT where you create recipes for simultaneously posting on a number of these platforms.

Remember, it’s always a good practice to tag the various URLs you use to drive traffic from various campaigns on referring sites. You can use the standard URL builder which google provides to generate tags.

By generating campaign URLs, you can identify the source of referrals to your website, whether visitors found the link from within a newsletter, social media post or other marketing campaigns. By naming the three main campaign tagging elements:  source, medium and campaign, Google Analytics will display information about where the referral originated. Simply complete the tool’s three-step form.

Here are just a few examples of valuable KPI data points you might consider tracking as part of acquisition:

  • Organic Search (SEO)
  • Paid Search Marketing (SEM)
  • Social Campaigns
  • Banner Campaigns
  • Links from External Sites
  • Links from Online Videos
  • Email Recipients
  • RSS Subscribers

Another important parameter which you would want to track is the landing page and how you can optimize them for better conversions. Google analytics helps you identify the most important landing pages on your site and the user flow thereafter. This would give you a better understanding on which pages are performing badly and helps you understand what you can do to further improve user interaction on those pages. [Behavior » Site Content » Landing Pages or Content Drill Down ]

image

On Improving weak landing pages:

  • Optimize the content to make it relevant if it’s outdated.
  • If it’s your main landing page, change the message or positioning if required. Use the heatmap tool to better understand the user interaction on the pages and optimize your page accordingly.
  • Make the content more comprehensive so that more people will find it interesting and informative.
  • Build more relevant internal links to the weaker pages to give them more link juice.
  • You can prompt the user to sign-up for email newsletters or at least try and convert them on any of your micro-conversions before the user leaves.

Middle of Funnel:

Middle of Funnel in the Digital Marketing Funnel is the point where in the user is moving from an initial product or brand interaction to a first sale/ to any major interaction on the platform. You might not be able to get a user to convert during this stage but it’s crucially important for companies to target micro-conversions during this stage.

It’s important to track the sources or channels through which the users come back to your site during this stage and it’s also important to measure the paths taken by the users in completing the micro-conversions or goals set on your page. For understanding user paths, GA has an option called Visitor Flow under Audience that visually represents the user path on the website and the drop-offs at each stage. The Visit Flow Report is a nice and a better representation of the traditional click path report. One can view the visitors moving between nodes. One also has the option to view particular segments of users based on region, campaign, traffic source, country etc and their flow/ browsing pattern on the website.

image

You can also create your own funnel for any of the goals you have set using GA to better understand where the users are dropping off. For setting up goals or micro-conversions in your site, you would need to clearly define the business objectives for creating goals (micro-conversions). Few examples of good engagement goals to track:

  • Account signup
  • Email signup
  • RSS subscription
  • Watching video
  • Content interactions (e.g. photo zoom, faceted search attributes, etc.)
  • Product Purchase

The goals would vary based on the type of website you are measuring for. To set up these goals, you can login in to the admin panel of your Google Analytics dashboard and then click on the Goal tab.

image

You have different goal types to chose from: Destination, Duration, Pages/ Screens per visit or Event. In case of an E-commerece website for eg, if the marketer needs to track how many users complete the check-out process, then he/ she would have to chose the type of the goal as “Destination” in the first step. In the second step he/ she would have to define the destination page which would complete the goal (Conversions).

image

For creating the funnel, you would need to specify each step (page) the user traverses before completing the final goal. The funnel visually represents each stage in the micro-conversion process also specifying the drop-offs at each stage. You can create, based on your requirements, multiple mini-conversions and funnels to better understand user flow during this middle stage of user lifecycle.

image

[Fig: A funnel representation of a goal set to White paper Downloads from the start page clearly indicating the conversions and drop-offs at each stage.]

In the middle of the funnel (MOF) for the Digital Marketing Funnel, it’s also important to analyze the most effective and popular channels that bring the user back. For this, GA provides Multi-Channel attribution tools under the “Conversions” section. There are various attribution models one could use. For a full guide refer this. The Linear Attribution Mode, which gives equal weightage to any channel in the funnel irrespective of where it appears,  gives us great insight in to which channel accounts for the most revenue overall. You can use the Model Comparison Tool in GA to find this out:

image

For figuring out the most popular channels in the MOF, we would have to do some manipulation using excel to weed out the first and the last interaction channels.

Bottom of the Funnel:

The bottom of the funnel is the last touch before someone buys. These channels are very important as it let’s you identify which channels to focus on to complete conversions. You can find this data in Conversion > Attribution > Model Comparison Tool and select your model as the Last interaction.

image

You can use these data on the best channels for driving traffic on to your website to further improve and optimize.

Segmenting:

In addition to standard segments that are available in GA to chose from ( You would have noticed this when we discussed the User Flow path), there are also a wide variety of custom user segmenting options that lets you better understand each set of users. You can create your own segments from the dashboard by clicking on the drop-down next to the All Visits tab that’s present as default. GA with the latest update now has the ability to segment visitors and not just visits, which is something GA lacked compared to tools like Kissmetrics and Mixpanel.

Now click on the Create Segments Icon to define your segments. There are a wide variety of parameters you can use to create segments or else you can use any of your own created events as well to define a segment.

image

image

Refer this post for a great list of custom advanced segments which you can use.

Using segments, you can slice and dice your audience in ways never imagined before. You can create segments based on first purchase value, browser being used, platform being used, device on which the visitor opened the site, purchase value during a period etc. I can very well use this data to do a cohort analysis which is very important at an early stage especially if you are on a lean methodology and constantly iterating, measuring the behavior of the set of users who come in during each of these iterations. Even otherwise, there is tremendous amount of insights analyzing segments will give you.

Understanding Cohort Analysis

Cohort analysis is conceptually pretty simple yet it’s one of the most important and powerful analysis approach a startup can adopt. I had in my earlier post discussed the importance of Lean Methodology for startups to minimize wastage of resources and getting to product/ market fit first before scaling up. Cohorts play a crucial role in helping us understand user behavior on each iteration or improvement to the product. There are plenty of other business questions that can be understood better using Cohort Analysis. To give you some examples:

1) How are the optimizations made to the product in a defined period affecting conversions?
2) Which traffic source is generating maximum conversions?
3) Which source tends to bring in users with maximum engagement on the platform?
4) Are customers acquired via email marketing more likely to repeat purchase or are they more likely to upgrade, compared to those acquired e.g. via AdWords marketing?

And more. Products such as Mixpanel and Kissmetrics enable us to easily create and analyze cohorts. Google Analytics in it’s early days did not have the Cohort Analysis feature, however, in 2017 they introduced the Cohort reports and according to me, this is one of the most powerful reports you can utilize in your analytics dashboard. And it’s FREE! 🙂

What is a Cohort?

A cohort is simply a group of people who share something in common and is time bound, ie, they had something in common when the grouping was first made. A Cohort is very similar to a segment and often there is a lot of confusion on the difference. To understand better, you can consider a segment as “Employees working in the Marketing Department” while a cohort would be more like “Employees who joined in November 2013”.

Cohort Analysis
Cohort Analysis is very popular in medicine where it is used to study the long term effects of drugs and vaccines:

A cohort is a group of people who share a common characteristic or experience within a defined period (e.g., are born, are exposed to a drug or a vaccine, etc.). Thus a group of people who were born on a day or in a particular period, say 1948, form a birth cohort. The comparison group may be the general population from which the cohort is drawn, or it may be another cohort of persons thought to have had little or no exposure to the substance under investigation, but otherwise similar. Alternatively, subgroups within the cohort may be compared with each other.
Source: Wikipedia

We can apply the same concepts for an online portal/ startup to understand better the different type of users and their behavior on the platform. How we define the cohorts to compare and what we compare about their behavior will depend on the business question we are seeking an answer for. In the case of a Lean Startup, the basic premise is that the product is constantly iterated to find the product/market fit and then iterated on to optimize conversions and scale. This is one of the prime applications of a cohort analysis. We can use Cohort Analysis to compare the users acquired during each iteration and compare their behavior on the platform in terms of retention, engagement, conversions etc. Joshua Porter’s excellent blog post on twitter’s use of Cohort Analysis to track engagement with product improvements is a great example of this.

image

If you look at the fig, it has rows for cohorts ( User acquired during each month is grouped as a separate cohort) and the columns give the engagement or retention figures for the cohort over a 12-Month period. As you can see this is the only manner in which one could clearly understand if the iterations and product improvements which twitter was rolling out on a regular basis was continually improving the engagement on the platform. Under a normal graph where in the cohorts are not present, many a times this picture won’t get reflected as the engagement from the early set of users will mask the engagement metrics of a particular group, be it in a negative or a positive manner.

The above example from twitter represents just one application of Cohort analysis. There are various business questions as discussed earlier that can be answered using cohorts. Let’s first understand the various ways to define cohorts:

1. Cohorts defined by when the user first Visits:
Many a times a user does not sign up or engage the first time they visit a platform. Grouping users based on their first visit will help one to understand the number of touches required before they sign up or engage on the platform and on what product iterations does one increase the conversion or the engagement metric based on the date of first visit. The earlier case study of Twitter is a good example of using cohorts to understand user engagement for a product.

2. Cohorts defined by when the user Converts:
By Converts, I mean any type of conversion or micro-conversion on the platform. It could be signing up, registering, making a first purchase, subscribing to the list etc.

3. Cohorts define by what channel the user was acquired on:
It’s really important to understand the best channels of user acquisition and the behavior of the users acquired through each channel so that one can focus more on the channels that yield best results. Cohorts based on the Channel of acquisition helps in this.

4. Cohorts based on User behavior:
Users can also be grouped based on the behavior they exhibit on the platform. For eg: In case of Zoomdeck, there are users who are frequent visitors and infrequent visitors. Users can be grouped in to various cohorts based on their re-visit rate and engagement on the platform. This is important as it helps us better understand them by having a look at other metrics exhibited by them. For an e-commerce companies one would need to strategize differently for frequent buyers vs infrequent buyers and this can be done better through cohorts.

5. Cohorts based on Customer Lifecycle:
For a platform having a number of stages it’s important to track various metrics like retention, Customer Lifetime Value, Engagement etc. It could be a simple game having various levels and classifying users based on the levels they are in and understanding the various metrics exhibited by these cohorts would help one take better decision to incentivize the users and make them shift levels.

6. Cohorts based on User Characteristic:
There might be cases where one would also want to create cohorts based on certain user characteristics like Men Vs Women, The Country of Origin, Age Group etc to create targeted campaigns or provide customized incentives to improve the engagement, retention or revenue metrics exhibited by them.

We have covered in general the various cohorts that can be created, although I do agree there might be a few specific ones related to the niche you are operating in. Creating cohorts form just one part of the puzzle, the most important part is to use various metrics to understand the behavior exhibited by these cohorts which enables you to take business decisions. There are various metrics one would need to track depending on the niche, type of product and the product lifecycle stage the Product is in.

Metrics most often tracked between cohorts are:

1. Measures of User Engagement:
During the early stage of a product before validation, User Engagement (including activation) and Retention becomes two of the most important metric. Cohorts based on date of first visit/ conversion, enables us to understand how product iteration is improving user engagement or if any changes made to the product has negatively affected engagement. The earlier example of Twitter was about tracking engagement on the platform. Depending on the product you can define what user action is termed as engagement or activation on your platform.

2. Retention:
Just like engagement is important as a metric, any successful product should have good retention figures as well. I had covered the importance of retention and how it affects virality, cost of user acquisition and customer lifetime value in my earlier posts on Virality. Cohorts help us understand retention better by enabling us to accurately define what features and user flows are improving the retention numbers. Funnel tools don’t help us track retention which needs to record user activity over longer periods.

3. Customer Lifetime Value:
Customer Lifetime Value is probably the most difficult metric to track. One of the questions we might want to understand could be the channels of user acquisition that result in giving us the max. value for CLV, the particular activity that drives a user to upgrade plans, split-test different pricing plans to understand the optimum one, features or user flow changes that results in better CLV. All of these can only be understood better using a cohort group as it allows us to track a cohort over a period of time to better understand their behavior on the platform.

4. Measuring long life-cycle events:
A product undergoes many iterations and feature roll-out. It’s impossible to measure long lifecycle events using just funnels. A prime example could be measuring revenues or retention which is typically a long term thing.

Now depending on the niche and the stage of growth your startup is in, you would have to choose the various metric that you need to track and also for the various cohorts we had earlier described. At the end of the day for any product, things finally boil down to user growth, engagement, retention and revenue. Analytics enable us to improve on each of those metric and cohort analysis is a technique that gives us great insights in measuring metric that are typically long cycle.

Cohort Analysis Presentation (Example)

I love this presentation of Cohort analysis (quoted from this Blog post) :

image

What you can see immediately is that the area on the right (Period 5) stacks up the current status with users from Period 1 to Period 4. The really interesting piece of the puzzle comes into play when you are considering what exactly your users represent: active, subscribers, etc. So here is what we can infer from the chart:

  • The height of the chart at Period 5 (at 280) is the number of users currently using (or paying for) our system/app.
  • The individual stacks have a drop-off. As we can see, the drop-off is high in the beginning and then starts to level out but does not go down to zero. Since this is homogeneous across all periods, we can infer that there is something we are doing right: user behavior becomes predictable.
  • For each period 1 to 4, new users were signing up and the number of users from Period 1 makes up 17.8% (50 out of 280) of the users in Period 5.
  • The fall off of users from one Period to the next is higher in subsequent Periods, leveling out at about 25%  of the original sign-ups after 3 periods.

References: