Merging survey waves and participant files: a guide for researchers
By the Analistable team · Updated · 2 min read
Research data arrives in waves, sites and instruments, each as its own file, joined on a participant ID. Before analysis you need to stack waves, link files and check for unmatched or duplicated IDs. Analistable lets you explore and check those joins in the browser; for the final analysis pipeline, scripted merges in R, Stata or Python remain the reproducible choice.
The multi-sheet jobs researchers actually do
- Stacking survey waves: combine quarterly or annual waves where questions were added, renamed or dropped.
- Linking instruments: join a demographics file to survey responses and outcome measures on participant ID.
- Checking the join: find IDs present in one file but not another, and duplicate IDs that would multiply rows.
- Matching responses to a sample frame: see who in the invited sample responded.
- Reshaping wide exports: unpivot wide question-per-column files before merging them with long data.
| File | Typical source | Join key |
|---|---|---|
| Survey wave export | Survey platform | Participant ID + wave |
| Demographics | Recruitment sheet | Participant ID |
| Outcome measures | Lab or clinic export | Participant ID + visit date |
| Sample frame | Invitation list | Participant ID or email |
Check consent and ethics terms before loading identifiable data into any tool. Analistable processes rows in your browser and does not store them, but pseudonymised IDs are still the safer default.
How Analistable helps
Load the files and ask "which participant IDs are in wave 2 but not wave 1?" or "how many duplicate IDs does the outcomes file have?". The AI writes SQL you can read, which makes it easy to carry the same logic into your analysis script. It is best used as a fast check and exploration step; it is not a statistics package and does not replace a documented, scripted pipeline.
Start here
Frequently asked questions
- Can I export the merged data for analysis?
- Analistable shows the SQL behind every answer and can save charts as images. For a reproducible record of your merge, copy that SQL (or the same logic) into your R, Stata or Python script rather than relying on a manual step.
- Will it handle large files?
- The in-browser database handles hundreds of thousands of rows on a typical laptop; very large datasets are better merged in a script or database.
- Does it run statistical tests?
- No. It answers counting, grouping and matching questions with SQL. Use your statistics software for modelling and inference.