One9Six / Working playbook
A test plan for hospitality performance marketing
Give a campaign test a fair chance to teach you something. Define the change, the comparison and the result that would justify putting it into practice.
One9Six / Working notes
Write the decision before the test
A useful test distinguishes a commercial hypothesis from a change in reported attribution. Choose one primary outcome and a minimum commercially meaningful improvement before seeing results. There is no universal number of days or bookings that guarantees a reliable answer.
| Focus | Record | Practical check |
|---|---|---|
| Hypothesis | A clearer cancellation message increases completed bookings | Keep price, room availability and traffic allocation comparable |
| Primary outcome | Completed bookings per eligible visitor | Predefine exclusions and the unit of randomisation |
| Decision rule | Commercial effect, uncertainty and operational impact | Report inconclusive results when evidence cannot support a decision |
Apply this to your next project
- Account for weekday mix, booking lead time and seasonality.
- Avoid stopping as soon as a favourable result appears.
- Check cancellation and margin outcomes when the booking cohort matures.
Make it practical
A useful place
to start.
Use this guide with the colleagues involved in the work. Note what you know, what needs checking and who will take the next step.
Download PDF- Write one hypothesis. State the audience, intervention, comparison and expected business effect. Example: a clearer family room landing page will increase completed bookings from the same eligible traffic. Do not bundle a new offer, new bidding and new creative into a single unexplained test.
- Choose a comparison you can trust. Prefer a supported randomised experiment when feasible. Record assignment, eligible traffic and possible overlap. A before and after comparison during changing seasonality provides weaker causal evidence. A geography split also needs sufficient independent markets and checks for spillover.
- Define the primary outcome. Choose completed bookings or an agreed contribution measure. Use click through rate, booking engine starts and engagement as diagnostics. Record whether the test measures a creative comparison, attributed campaign performance or incremental demand. These are different questions.
- Check how much data the test needs before you start. Use the baseline conversion rate or outcome variability, the smallest commercially useful effect and the available volume. Set the evaluation window with an analyst. There is no universal seven day duration or conversion count threshold that makes every hotel test reliable.
- Control the commercial conditions. Record rates, inventory, minimum stays, cancellation policy and major local events. Avoid changing them asymmetrically during the test. Log unavoidable changes and decide whether they undermine interpretation.
- Set safety and decision rules. Specify maximum spend, unacceptable booking errors and the minimum effect worth deployment. Distinguish an emergency stop from ending a test because an early result looks attractive. Repeatedly checking significance requires a statistical method that accounts for it.
- Give the results time to become clear. Allow for conversion delay and relevant cancellations. Report the effect estimate and uncertainty, not only a winner label. If the sample cannot distinguish the useful effect from noise, classify the test as inconclusive.
- Use the result to decide what happens next. Record rollout, retest or rejection, the evidence behind it and the person responsible. Check performance after rollout because traffic mix, competition and inventory can change. Preserve unsuccessful tests as useful evidence.
The method in practice
Test card: a family room proposition
Illustrative working example. Figures and decisions are not client results or market benchmarks.
Intervention
Change only the landing page explanation of sleeping arrangements and included services. Keep the price, eligible room, traffic source and cancellation terms consistent. Randomly assign eligible visitors if the implementation supports it.
Success and limits
Primary outcome: completed bookings per eligible visitor. Diagnostic: availability search starts. Guardrails: booking errors and cancellation quality. An improvement identifies the page effect within this test population, not the incremental value of the entire advertising budget.