Data Sources

ILR data

• Start of the process. Surveys are generated from it. Every response matches back to it.
• Location. A WYCA SharePoint library, in WYCA's own tenant. Keeps Fabric reading in-tenant and avoids cross-tenant access.
• Access. A service account with read access. Set up and held by WYCA / Jamie.
• Format. Fixed Excel layout, per the ILR Data Interface spec. One row per learner-aim. Same columns for every provider, and for dummy and real.
• Category. Derived from the aim reference. Not typed in by hand.
• To confirm. Exact library path. Service account name. Cadence (monthly, full vs incremental).

Surveys

• Grain. One baseline and one outcome survey per learner. Not per aim. No course questions.
• Per provider. A separate survey for each provider. Around 15 in total. Each has its own survey ID and participant list.
• Matched pair. Baseline joined to outcome on the learner reference. Within a provider, then unioned. Built in the model, not reliant on Snap seeding.
• Aims. Come from the ILR, joined to the learner. A learner counts once. Course and level are learner attributes.

API calls

• GET /surveys — lists surveys and totals. Used to find survey IDs.
• GET /surveys/{id} — one survey's detail.
• GET /surveys/{id}/variables — the questions and codes. The response schema.
• GET /surveys/{id}/participants — who was invited and their status. Gives surveys sent out and the response rate.
• GET /surveys/{id}/responses — the answers. Charged 0.1 unit each.

API access

• One credential. A single pair from IBP: X-USERNAME (account email) and X-API-KEY.
• One account. It must have access to every pilot survey, or one key won't cover them all.
• Service account. A dedicated login, not a personal one. Stored as a secret, never in code.
• Units. The account needs a unit balance for the charged response calls.

Knowing what's new

• Progress token. Each responses call returns a token marking where it reached.
• First run. startingFrom=0 pulls everything.
• Later runs. Pass the stored token. Only new or changed responses come back.
• Fabric. Keeps the latest token per survey. Keeps cost and traffic down.

Needed from IBP

• Credentials. The API account email and key (X-USERNAME / X-API-KEY), from one service account with access to every pilot survey, with a unit balance.
• Survey lookup table. A small reference table (around 15 rows) mapping each survey ID to what it is. IBP provide it as a static file, since they create the surveys and own the IDs. It is kept in the Lakehouse for now (inside WYCA's tenant, so no cross-tenant issue), and IBP reissue it if surveys are added. Columns:
  • Survey ID — the Snap survey ID
  • Provider — e.g. Bradford College
  • Wave — baseline or outcome
  • Route — email or QR
• Naming convention (nice to have). If surveys are also named consistently, e.g. Baseline | Email | Bradford College, GET /surveys can cross-check the table. The table stays the source of truth.
• Learner reference. Held as a hidden variable in both surveys.

QR / generic route caveat

• No participant. Learners self-identify. There is no known participant and no reliable key.
• Effect. No response rate (no denominator). Pairing and ILR matching are fuzzy for that route.