CXL Institute: Growth Marketing Mini Degree Review #6️⃣
Qualitative Research, Survey Design Theory, Cognitive Biases, and Common Mistakes.
In last week’s article review of the CXL Growth Marketing mini degree, we have talked about the optimization process and the research model that involves Heuristic Analysis, Technical Analysis, Digital Analytics, Qualitative Research.
Each analysis is pivotal to understand your main roadblocks, how to identify elements of your website that provoke the most leakage of money and decrease your conversion rates.
If you have missed the previous article, check it out 👉 here.
In this article, I wanted to have an in-depth review of what it takes to run successful qualitative research, and how this is important in every step of the way towards your growth marketing strategy. The logic is simple if you do not know your customers, how are you supposed to gain their trust, or attempt to solve their problems and pains?
Let’s dive together with one of the most influential market researcher, Dr Rob Balon, he guides us on how to design surveys that can provide gold insights for optimizers and marketers.
Why Surveys Play Such An Important Role For Your Optimization Process?
A survey can be defined as a sub-sample or cross-sample of the general population. A well-structured survey allows marketers to receive feedback on their product but especially it serves to:
- Understand attitudes towards your brand
- Stay aware of competitors
- Evaluate new products
- Better understand our customers
N.B. Keep in mind that with a properly conducted survey you will have always a +/- 5% error margin.
Survey Design Strategies
Giving his extensive professional experience as a marketer researcher, Balon outlines the importance to know about your competitor as well as you know about your company. Getting their heads, understand what they are doing, and try to get their customers switching to your product/service.
One survey design strategy to find out where your customer base is, to run the survey with your competitors’ customers with the addition of only three or two questions about the customers’ perception of your own brand.
When designing a survey you have a limited number of questions, hence it is important to give focus to the most urgent aspects you want to investigate.
A piece of general advice is to insert in your survey:
- Questions that are closed-ended
- Questions that are not asked a leading way, meaning that you design a question that prompts or encourages the answer wanted.
- Directly address what you want to ask, get to the heart of the information you want to achieve.
Regarding the sample size in online testing, generally, 20 people are not enough, and the larger the sample the smaller the margin error of your data analysis.
5 Common Mistakes in Survey Design
Error#1 — Non-intuitive scale
Do not confuse your respondents!
For example, the highest number on a scale should always be always the highest value.
Error#2 — Mixing behavior and attitude questions
Attitude ≠ Behavior
Social psychologists have found that attitude and behavior are not perfectly aligned as many would think. Below a table where Joshua Miller lays out the main differences in this article:
Error#3 — Questions That Do Not Communicate
Do not ask irrelevant questions to your target audience.
Avoid use jargon or technical verbiage that your audience cannot understand.
This is important because whenever your respondents do not understand what it is been asked, they might as well leave the survey incomplete.
Error#4 — Too Long Survey
On average a good survey should take between 5–10 minutes to complete. This is because, on average the longer the survey, the more fatigue is experienced by the respondents and the risk of error of central tendency can be higher.
This error is defined as the tendency of respondents, affected by fatigue, to answer the questionnaire giving the same type of answer (i.e. all within the same given range) and when this happens, you know you lost them.
Error#5 -Not Keeping a Neutral Learning Curve
Users learn from your surveys and, while filling it, they might become biased in their answers, meaning that on average a respondent can be biased along with the survey because they grasp what type of answers you want them to choose, so the general advice here is to structure the survey in a neutral way for each question.
In-House Surveys? Keep this in mind!
When conducting surveys with those customers that had already a relationship with your company, you need to be aware of the fact that there is a certain bias on the fact that they are either pre-disposed positively or negatively toward your company, based on the previous experience they had.
Keep it short, on average a good survey contains between 10 -15 questions, and for an existing customer, the sample size should be a minimum of 100. When designing this type of survey remember to not assume that your customer has a product-knowledge as you, marketer, have.
Cognitive Biases in Survey Design
How to recognize common biases in yourself, as a researcher, and how can you minimize the bias-effect when analyzing your data?
One dangerous bias can be the lack of communication between the researcher and the customer, hence when the researcher tells the client what they believe the client wants to hear, also called in the field as reading the room.
Another dangerous bias is the order bias, hence this generally occurs when the items listed higher in the survey receive better scores. When this happens, a way to minimize this bias is to change the order, rotate the questions, and give a totally different order.
Cognitive Biases on the User End
When designing a survey, it is important to recognize any space that can produce cognitive biases at the user end.
Balon provides two key suggestions to avoid this type of bias:
- Write the survey in an effective executive summary in a way approachable to your customer, avoid technical jargon. According to SurveyGizmo, it can be defined as:
“An executive summary is a document that efficiently summarises a larger business plan while communicating key findings and takeaways from the research, as well as proposed courses of action.”
2. Another important piece of advice is to debrief your customer ahead of time.
Be aware of selective perception, meaning that customers will have the tendency to pay more attention to what they already agree with.
Use your customers’ voice
A qualitative survey is great in providing you with information about the lexicon of your customers, what type of verbiage they use, and how you can include it with your survey. Another perk is to include the same lexicon/verbiage in your copywriting, in order to increase the conversion rates of certain key pages on your website, or simply including key terms used by your customers in your e-mail marketing efforts.
This is extremely important because whenever you align with your customer’s voice you can performing better surveys while increasing the level of communication between you and your customers.
🔥 Want to know more about growth marketing? 🚀
👉 Stay tuned in the next article we will talk about the principles of A/B Testing and the popularity that this method gained across the most influential and successful marketing agencies.
