Week 04 · Marketing Research & Analytics
Davood Wadi, PhD
The quality of survey findings depends on the quality of the measures and the questions from which they are derived.
Measurement is the assignment of numbers or other symbols to characteristics of objects according to specified rules.
The objects of measurement are consumers, brands, stores, or advertisements. What is measured is a characteristic of these objects, such as a consumer's income or a brand's perceived quality.
Scaling is the creation of a continuum on which measured objects are located, such as a scale from 1 to 7 on which consumers rate a brand.
The rules of assignment must be consistent, so that the same characteristic always receives the same number.
Many characteristics of interest in marketing are constructs: concepts that cannot be observed directly, such as satisfaction, brand loyalty, or purchase intention.
A conceptual definition states the meaning of a construct in words.
An operational definition specifies the procedure by which the construct is measured, including the questions asked and the response options offered.
For example, customer satisfaction may be operationally defined as the average of three ratings of satisfaction with a product, each on a scale from 1 to 7.
A single construct can be operationalized in several ways, and the choice of operational definition affects the findings.
Every measure has a level of measurement: nominal, ordinal, interval, or ratio.
The level of measurement determines which statistical analyses are appropriate.
Each level possesses the properties of the levels below it, together with an additional property.
A nominal scale uses numbers only as labels to identify or classify objects.
Examples include respondent identification numbers, gender categories, and region of residence. Assigning the number 1 to one region and 2 to another implies no order between them.
An ordinal scale indicates the relative position of objects, but not the magnitude of the differences between them.
Examples include the ranking of brands by preference and categories of education.
A consumer who ranks three brands first, second, and third reveals an order of preference, but not how much more the first brand is preferred to the second.
An interval scale has equal distances between adjacent points, so that differences between values can be compared.
The zero point of an interval scale is arbitrary. Temperature in degrees Celsius is an interval scale: 20 degrees is not twice as warm as 10 degrees.
Rating scales, such as a scale from 1 to 7, are commonly treated as interval scales in marketing research.
A ratio scale has all the properties of an interval scale, together with an absolute zero point.
Examples include income, age, sales in units, and market share. A customer who spends 200 dollars spends twice as much as a customer who spends 100 dollars.
Nominal data permit counts, percentages, and the mode.
Ordinal data additionally permit the median and percentiles.
Interval data additionally permit the mean and the standard deviation.
Ratio data additionally permit statements of proportion, such as “twice as much.”
An analysis that assumes a higher level of measurement than the data possess produces results that cannot be meaningfully interpreted, such as the average of respondents' region codes.
Sort each analysis by whether the level of measurement of its data permits it.
Tap a card, then tap the group it belongs to.
Scaling techniques are classified as comparative or noncomparative.
In comparative scales, respondents compare objects directly with one another.
In noncomparative scales, each object is evaluated independently of the others.
In a paired comparison scale, respondents choose one of two objects according to a stated criterion, such as preference.
In a rank-order scale, respondents rank several objects according to a criterion, such as ranking five coffee brands from most to least preferred.
In a constant sum scale, respondents divide a fixed number of points, typically 100, among several attributes or objects to indicate their relative importance.
Comparative scales produce ordinal data, with the exception of constant sum scales, which are often treated as interval data.
Comparative scales detect small differences between objects, but their results cannot be generalized beyond the objects included in the comparison.
A Likert scale presents a series of statements, and respondents indicate their degree of agreement or disagreement with each.
Response categories typically range from “strongly disagree” to “strongly agree,” with five or seven categories.
For example: “This bank's mobile application is easy to use,” rated from 1, strongly disagree, to 5, strongly agree.
The ratings for several statements measuring the same construct are summed or averaged to produce a single score.
Likert scales are easy to construct and to administer, but respondents must read each statement carefully, which increases the time required to complete the questionnaire.
A semantic differential scale presents pairs of adjectives with opposite meanings at the two ends of a scale, typically with seven points.
Respondents indicate the point between the two adjectives that best describes the object.
For example, a hotel may be rated on scales between “modern” and “old-fashioned,” “friendly” and “unfriendly,” and “expensive” and “inexpensive.”
Semantic differential scales are widely used to compare the images of brands, stores, and companies.
The adjective pairs are presented with the favorable adjective on the left for some pairs and on the right for others, so that respondents do not answer by habit.
Each case describes a survey item. Name the scaling technique it uses.
Pairs of opposite adjectives at the two ends of a scale, with the favorable adjective on different sides: a semantic differential scale.
A fixed number of points divided among attributes to show their relative importance: a constant sum scale.
A statement rated by degree of agreement, evaluated on its own: a Likert scale.
A choice of one of two objects according to a criterion: a paired comparison scale.
Several objects placed in order by a criterion: a rank-order scale.
Opposite adjectives at the ends of a seven-point scale, used to describe the image of a company: a semantic differential scale.
Which concept does this case show?
The number of categories: more categories permit finer distinctions, but respondents can reliably distinguish only a limited number. Five to nine categories are commonly used.
Balanced or unbalanced: a balanced scale has an equal number of favorable and unfavorable categories.
Odd or even number of categories: an odd number provides a neutral midpoint. An even number requires respondents to lean in one direction.
Forced or nonforced: a nonforced scale includes a “no opinion” or “don’t know” option, which is appropriate when some respondents lack the knowledge to answer.
Verbal labels: labels may be provided for every category or only for the endpoints.
Each case describes a change to a rating scale. Name the decision the researcher has made.
A “don’t know” option makes the scale nonforced, which suits respondents who lack the knowledge to answer.
An even number of categories removes the neutral midpoint and requires respondents to lean in one direction.
The favorable and unfavorable categories are unequal in number, so the scale is unbalanced.
Labels are provided only for the endpoints rather than for every category.
Respondents can reliably distinguish only a limited number of categories; five to nine are commonly used.
Which concept does this case show?
Constructs are usually measured with several items rather than a single question.
Each item captures part of the construct, and the combined score measures the construct more accurately than any single item.
Many multi-item scales have been developed and validated in published research, for constructs such as satisfaction, brand attitude, and service quality.
A validated scale is used in its published form. Changing the wording of its items can change what the scale measures.
Every measure contains some error.
The observed score equals the true score plus systematic error plus random error.
Systematic error affects every measurement in the same way, such as a question that consistently encourages favorable answers.
Random error varies unpredictably from one measurement to the next, such as errors caused by a respondent's fatigue or distraction.
Reliability and validity are the criteria by which the extent of these errors is evaluated.
Reliability is the extent to which a scale produces consistent results when the measurement is repeated. It reflects the absence of random error.
Test-retest reliability is assessed by administering the same scale to the same respondents at two points in time and correlating the results.
Alternative-forms reliability is assessed by administering two equivalent versions of the scale and correlating the results.
Internal consistency reliability is assessed by examining whether the items of a multi-item scale produce consistent results.
Coefficient alpha, also known as Cronbach's alpha, is the most widely used measure of internal consistency. A value of 0.70 or higher is conventionally regarded as acceptable.
Validity is the extent to which a scale measures the construct it is intended to measure. It reflects the absence of both systematic and random error.
Content validity is the extent to which the items cover the full domain of the construct, as judged by experts.
Criterion validity is the extent to which the scale is related to other variables with which it should theoretically be related, such as actual purchases.
Construct validity is the extent to which the scale behaves as theory predicts.
Convergent validity is shown when the scale correlates with other measures of the same construct. Discriminant validity is shown when it does not correlate strongly with measures of different constructs.
A measure can be reliable without being valid.
A bathroom scale that consistently reads three kilograms too high is reliable, because its readings are consistent, but it is not valid, because its readings are systematically wrong.
A measure cannot be valid without being reliable, since inconsistent results cannot accurately reflect the true score.
Reliability is therefore a necessary but not a sufficient condition for validity.
Sort each measure by what it achieves.
Tap a card, then tap the group it belongs to.
A questionnaire is a structured set of questions designed to obtain information from respondents.
It translates the information requirements of the research problem into questions that respondents are able and willing to answer.
It also motivates respondents to complete the questionnaire and minimizes response error.
Open-ended questions allow respondents to answer in their own words. They provide rich information but require coding before analysis.
Multiple-choice questions offer a list of response options, from which respondents select one or more.
Dichotomous questions offer only two response options, such as yes and no.
Scaled questions use the rating scales introduced earlier in this week.
Structured questions are faster to answer and to analyze, but they limit responses to the options the researcher has anticipated.
Questions use words that respondents understand, and avoid technical terms.
Questions avoid ambiguous words, such as “often” or “regularly,” whose meaning varies across respondents.
Questions avoid leading wording that suggests the desired answer, such as “Do you agree that our new menu is an improvement?”
Questions avoid double-barreled wording that asks about two issues at once, such as “How satisfied are you with the price and quality of the product?”
Questions avoid implicit assumptions, such as asking how often a respondent uses a feature without first establishing that the respondent uses it.
Questions ask only for information that respondents are able to provide, such as recent rather than distant purchases.
Each draft question breaks one wording principle. Name the problem.
It asks about two issues at once, so one answer cannot show which one the respondent means.
“Regularly” may mean weekly to one respondent and yearly to another.
The wording suggests the desired answer.
“Omnichannel fulfillment proposition” is a technical term that respondents may not understand.
It assumes that the respondent uses the feature without first establishing it.
Respondents cannot accurately recall a distant purchase; recent purchases are asked instead.
Which concept does this case show?
A questionnaire begins with simple and interesting questions that establish the respondent’s cooperation.
Screening questions, which establish whether the respondent qualifies for the study, appear at the start.
The funnel approach moves from general questions to specific questions, so that specific questions do not influence answers to general questions.
Sensitive questions and classification questions, such as income, are placed near the end.
Order effects occur when earlier questions influence answers to later questions. Rating a brand's safety, for example, can raise the importance that respondents later assign to safety.
The order of response options can also affect answers, and is therefore randomized where possible.
Response bias is the systematic tendency of respondents to answer in a way that does not reflect their true opinions or behavior.
Social desirability bias is the tendency to give answers that present the respondent favorably, such as overstating recycling or understating alcohol consumption.
Acquiescence bias is the tendency to agree with statements regardless of their content.
Extreme responding and midpoint responding are the tendencies to select the endpoints or the middle of a scale regardless of the question.
Interviewer bias occurs when the presence or behavior of an interviewer influences responses.
Response bias is reduced through careful wording, balanced scales, assurances of anonymity, and self-administered questionnaires for sensitive topics.
Match each form of response bias with the case that shows it.
Response bias
Case
Tap a term, then the example it goes with.
Pretesting is the administration of the questionnaire to a small sample of respondents from the target population, in order to identify and eliminate problems before data collection.
In cognitive interviewing, respondents explain aloud how they interpret each question and arrive at their answers.
Pretesting reveals ambiguous wording, unclear instructions, missing response options, and questions that respondents are unable or unwilling to answer.
The questionnaire is revised after the pretest, and a substantially revised questionnaire is pretested again.
AI tools are now widely used to draft, revise, translate, and review survey questions.
They reduce the time required to produce a first draft of a questionnaire.
The principles of measurement introduced in this week apply to AI-generated items in the same way as to items written by researchers.
AI can be used to generate candidate items for a construct, to propose response options, and to produce alternative wordings of a question.
AI can be used to review a draft questionnaire for leading, double-barreled, or ambiguous wording, and to translate it into other languages.
AI-generated items may appear appropriate while failing to cover the full domain of the construct, which reduces content validity.
AI tools may present items as coming from a published, validated scale when the items have been altered or invented.
AI tools may also rephrase the items of a validated scale, which changes what the scale measures.
AI-simulated respondents can be used to answer a draft questionnaire and to report which questions they find unclear, before the questionnaire is shown to real respondents.
This procedure can identify some ambiguous wording and missing response options at low cost.
AI-simulated respondents do not share the knowledge, experiences, and interpretations of the target population, and cannot reveal all the problems that real respondents would encounter.
AI pretesting therefore supplements pretesting with real respondents and does not replace it.
The researcher first defines the construct conceptually.
The researcher then searches for an existing validated scale, and uses it in its published form if one is suitable.
If no suitable scale exists, AI is used to generate a pool of candidate items.
Experts then review the items for content validity and for flaws in wording.
The revised items are then pretested with AI-simulated respondents to identify obvious ambiguities.
The items are then pretested with real respondents through cognitive interviews.
Finally, the scale is administered to a pilot sample, and its reliability and validity are assessed.
A research team needs a scale to measure trust in a food-delivery app. Put the steps of the procedure in order.
Tap a step, then the number where it belongs.
AI models that advise and shop for consumers may hold systematic evaluative tendencies, which researchers measure with the same multi-item scales used for consumers.
The scale is administered in its exact validated form, so that the measure retains its established reliability and validity.
For example, the 17 items of the Consumer Ethnocentrism Tendencies Scale (CETSCALE) have been administered to language models, with the name of the country varied across conditions (Wadi, Ghodrat, & Philp, 2026).
A fixed instruction requires the model to answer with a single scale point, because open-ended answers can be evasive, such as "As an AI, I do not have a country."
Each item is administered separately, with the name of the country inserted into the wording of the item.
A research team wants to measure the price consciousness of three language models that a retailer is considering as shopping assistants. A validated eight-item price-consciousness scale exists. Which procedure follows the approach introduced on this slide?
A language model does not hold one fixed answer to a Likert item. It assigns a probability to each point of the scale, and a generated answer is one draw from this distribution.
A single generated answer can therefore misrepresent the model's position on the item.
With open-weight models, the researcher can read the probability that the model assigns to each scale point directly, and so obtain the exact response distribution without repeated sampling (Wadi, Ghodrat, & Philp, 2026).
The distribution shows both the model's typical response and its certainty, which a single answer cannot show.
Discussion: A product manager uses an AI tool to produce a complete customer satisfaction questionnaire in ten minutes and proposes to launch it the same day. Which checks should the questionnaire undergo before launch, and which of them cannot be performed by an AI tool?
Measurement assigns numbers to characteristics of objects according to rules, and constructs are measured through operational definitions.
The level of measurement, whether nominal, ordinal, interval, or ratio, determines which statistical analyses are permissible.
Comparative scales compare objects directly; noncomparative scales, such as Likert and semantic differential scales, evaluate each object independently.
Reliability is the consistency of a measure, validity is the extent to which it measures the intended construct, and a measure can be reliable without being valid.
Question wording, question order, and response bias affect the accuracy of responses, and every questionnaire is pretested before data collection.
AI accelerates item development and review, but AI-generated items must be assessed for content validity and pretested with real respondents.
Validated scales can be administered to AI models in their exact form, and a model's response to an item is a distribution over the scale points rather than a single answer.