04. Pilot translations, instruments and interviewer workflow
Lesson
LEARNING OBJECTIVES
Plan a pilot that tests the whole workflow, identify problems in translation and comprehension, and train interviewers using observable performance criteria.
PILOT THE PROCESS, NOT JUST THE FILE
An electronic form can pass technical checks while participants misunderstand its questions. Test recruitment, consent, eligibility, wording, translation, skip logic, measurement, data recording, synchronization where relevant, supervision and daily review.
Recruit pilot participants appropriate to the aspects being tested, including relevant languages and accessibility needs. A small feasibility pilot does not establish population prevalence or validate every psychometric property of an instrument.
TRANSLATION AND COMPREHENSION
Use a documented translation and review process suited to the instrument and context, respecting any permissions or established versions. Technical equivalence is not enough if a phrase has a different practical meaning. Ask people to explain what an item means and how they chose an answer; use neutral probes rather than teaching them the intended answer.
Back-translation can identify discrepancies but is not, by itself, proof of cultural or measurement equivalence. Record disagreements, decisions and who approved revisions. Keep versions synchronized across languages, training materials and programmed forms.
WORKED FICTIONAL PILOT FINDING
The phrase “main water source” is interpreted by some caregivers as the closest source and by others as the source used most often. A skip rule then sends households with multiple sources past an important exposure question.
Log these as separate comprehension and logic issues. Revise the definition and question path, test again and document the resulting variable interpretation. Do not simply instruct interviewers to improvise explanations differently at each visit.
TRAIN FOR CONSISTENCY AND RESPECT
Use role-play for consent, neutral questioning, eligibility ambiguity, refusal and unexpected distress. Assess whether interviewers follow the selection rule, read items consistently, apply probes appropriately and record responses correctly. Include equipment procedures, calibration where relevant and escalation rules.
Staff readiness should be demonstrated through observed practice. Attendance at a training session alone does not show competence. Supervisors also need procedures for feedback and correction without pressuring staff to produce particular answers.
YOUR TASK
Write a pilot plan with objectives, participants, tasks, observations, issue log, revision authority and retest criteria. Include one scripted mock interview with a refusal or unclear response. Set three observable readiness criteria.
Build an issue log with columns for issue, evidence, severity for the study, proposed change, decision, version and retest result. Decide whether pilot observations will enter the final dataset and justify this prospectively; do not pool them automatically after instruments change.
SELF-CHECK AND FEEDBACK
Can completion in ten minutes prove participants understood the form? No. Is translation review finished when two reviewers agree on spelling? No; meaning, usability and measurement intent matter. Does a pilot sample justify a precise prevalence estimate? Generally no; its purpose and selection are usually different.
PORTFOLIO OUTPUT
A pilot protocol, mock-interview checklist, versioned issue log and staff-readiness criteria.
FURTHER READING
WHO STEPS Manual: planning and field implementation resources.
https://www.who.int/docs/default-source/ncds/ncd-surveillance/steps/part2.pdf
CDC Field Epidemiology Manual: Collecting Data.
https://www.cdc.gov/field-epi-manual/php/chapters/collecting-data.html
Plan a pilot that tests the whole workflow, identify problems in translation and comprehension, and train interviewers using observable performance criteria.
PILOT THE PROCESS, NOT JUST THE FILE
An electronic form can pass technical checks while participants misunderstand its questions. Test recruitment, consent, eligibility, wording, translation, skip logic, measurement, data recording, synchronization where relevant, supervision and daily review.
Recruit pilot participants appropriate to the aspects being tested, including relevant languages and accessibility needs. A small feasibility pilot does not establish population prevalence or validate every psychometric property of an instrument.
TRANSLATION AND COMPREHENSION
Use a documented translation and review process suited to the instrument and context, respecting any permissions or established versions. Technical equivalence is not enough if a phrase has a different practical meaning. Ask people to explain what an item means and how they chose an answer; use neutral probes rather than teaching them the intended answer.
Back-translation can identify discrepancies but is not, by itself, proof of cultural or measurement equivalence. Record disagreements, decisions and who approved revisions. Keep versions synchronized across languages, training materials and programmed forms.
WORKED FICTIONAL PILOT FINDING
The phrase “main water source” is interpreted by some caregivers as the closest source and by others as the source used most often. A skip rule then sends households with multiple sources past an important exposure question.
Log these as separate comprehension and logic issues. Revise the definition and question path, test again and document the resulting variable interpretation. Do not simply instruct interviewers to improvise explanations differently at each visit.
TRAIN FOR CONSISTENCY AND RESPECT
Use role-play for consent, neutral questioning, eligibility ambiguity, refusal and unexpected distress. Assess whether interviewers follow the selection rule, read items consistently, apply probes appropriately and record responses correctly. Include equipment procedures, calibration where relevant and escalation rules.
Staff readiness should be demonstrated through observed practice. Attendance at a training session alone does not show competence. Supervisors also need procedures for feedback and correction without pressuring staff to produce particular answers.
YOUR TASK
Write a pilot plan with objectives, participants, tasks, observations, issue log, revision authority and retest criteria. Include one scripted mock interview with a refusal or unclear response. Set three observable readiness criteria.
Build an issue log with columns for issue, evidence, severity for the study, proposed change, decision, version and retest result. Decide whether pilot observations will enter the final dataset and justify this prospectively; do not pool them automatically after instruments change.
SELF-CHECK AND FEEDBACK
Can completion in ten minutes prove participants understood the form? No. Is translation review finished when two reviewers agree on spelling? No; meaning, usability and measurement intent matter. Does a pilot sample justify a precise prevalence estimate? Generally no; its purpose and selection are usually different.
PORTFOLIO OUTPUT
A pilot protocol, mock-interview checklist, versioned issue log and staff-readiness criteria.
FURTHER READING
WHO STEPS Manual: planning and field implementation resources.
https://www.who.int/docs/default-source/ncds/ncd-surveillance/steps/part2.pdf
CDC Field Epidemiology Manual: Collecting Data.
https://www.cdc.gov/field-epi-manual/php/chapters/collecting-data.html
My study notes
Stored only in this browser. Do not enter patient information.Saved in this browser. No account needed.