# Compare survey answers across collection modes

Check collection context before combining phone and web answers into a single survey conclusion.

If nothing was unclear or difficult, say so. Skip questions about steps you did not experience; use not applicable where appropriate.

## 1. How did you encounter the question in this pilot?

Answer: ____________________

## 2. What, if anything, did another person explain before your answer?

Answer: ____________________

## 3. Which part of the question was harder to revisit in this format?

Answer: ____________________

## 4. What would help you answer without feeling pushed toward a response?

Answer: ____________________

## Author notes / illustrative keys where applicable

1. Record mode as context without treating it as a causal assignment.
2. Identify interviewer explanation that can change the experienced question.
3. Surface a format limitation such as retaining spoken choices in memory.
4. Probe perceived pressure with room to say none.

Interpretation caution: These different people and collection conditions cannot isolate a causal mode effect; sample shares remain descriptive.


## Illustrative finding and follow-up

Observation: The two ten-person synthetic samples differ by two favourable answers; collection context and people also differ.

Action to test: Retain counts and mode notes; pilot the spoken and written presentation before combining them.

Follow-up: Check that revised phone and web introductions communicate the same question and response freedom.


## Collection protocol

When and whom to ask: Before mixed-mode collection, use the same fabricated question with volunteers in each intended format.

Decision owner: The method owner records recruitment and mode; the pilot facilitator checks the experienced question.

Follow-up: Check that revised phone and web introductions communicate the same question and response freedom.

Response handling: Do not record calls without appropriate consent. This worksheet uses invented totals and optional explanations.

## Decision workshop

Synthetic examples only. Manual worksheet; no automatic import or product writes.

### Synthetic pilot and reporting records

Mode | Valid answers | Favourable | Context
--- | --- | --- | ---
Phone | 10 | 6 | Interviewer present
Web | 10 | 4 | Self-completed

### Version changes

Boundary | Draft | Revised
--- | --- | ---
Interpretation | Mode caused improvement | Sample difference with context limits
Comparability | Same wording is sufficient | People, presentation and timing also reviewed

### Draft or earlier version (review its limits)

Phone collection improves satisfaction by 20 percentage points.

### Revised example

The synthetic phone sample is 6/10 and web sample 4/10 favourable. These descriptive samples differ in people and mode; neither establishes a causal mode effect.

### Decision situations

- People and collection conditions are documented: Describe both cohorts and their limits; do not infer a causal effect.
- Mode is mixed but not recorded: Hold mode-specific comparison until collection context is recoverable.
- A separate controlled design is proposed: Review assignment and analysis before making an experiment claim.

### Evidence dependencies

- Recruitment context → Bounded comparison: who answered
- Collection mode → Bounded comparison: how they answered

### Method references

- AAPOR survey best practices: https://aapor.org/standards-and-ethics/best-practices/ — Pretesting and transparent sample context.