Building a credible certification program means starting with a job task analysis, not with course content. You define the competencies the credential attests to, build an exam against them, set the pass mark through a formal standard-setting study, keep the certifying function independent from the training function, and require recertification so the credential stays meaningful.
The mistake that makes a credential worthless
Most organizations build a certification backwards. They start with content they already teach, write a quiz at the end, set the pass mark at seventy percent because that feels right, and issue a certificate. It looks like a certification. It will not withstand a challenge, and employers will eventually work out that it does not mean much.
A credential's entire value is that it reliably distinguishes people who are competent from people who are not. Everything below serves that one goal. The good news is that the rigorous path is not much slower than the sloppy one, and it produces something you can defend, accredit and charge properly for.
| This guide covers general credentialing practice and is not legal or accreditation advice. If you intend to pursue formal accreditation, work with a qualified psychometrician and your chosen accrediting body from the start rather than retrofitting later. |
Step 1: Run a job task analysis before anything else
A job task analysis, sometimes called a practice analysis, is the foundation the whole program rests on. It answers one question: what does a competent person in this role actually do?
- Convene subject matter experts who genuinely do the work, not only people who teach it.
- Document the tasks, the knowledge and the skills the role requires in practice.
- Survey a wider group of practitioners to validate and weight those tasks by frequency and criticality.
- Turn the results into a content outline, sometimes called a blueprint or exam specification, that says how much of the exam covers each domain.
This matters because it is what makes the certification job-role based rather than course-based. Accreditation bodies expect it explicitly. ANAB, for example, asks for a job or practice analysis for each certification submitted for accreditation, alongside an organizational chart and a defined scope. Skip this step and no amount of later polish will fix the foundation.
Step 2: Define the scheme, including who is eligible
The certification scheme is the formal specification of your credential. Under ISO/IEC 17024, the international standard for bodies certifying people, a scheme typically covers the scope of certification, the job and task description, the required competencies and abilities, any prerequisites, and a code of conduct where relevant.
- Scope: exactly what the credential attests to, and what it does not.
- Eligibility: education, experience or training prerequisites a candidate must meet before sitting.
- Assessment method: written exam, practical assessment, portfolio, or a combination.
- Recertification: what holders must do to stay certified, and how often.
- Code of conduct and grounds for revocation, where the credential carries professional responsibility.
Write this down formally before you build anything. It becomes the document you point to when a candidate disputes a decision.
Step 3: Build the exam properly
- Write items against the blueprint, so each question maps to a competency the job task analysis identified. Subject matter experts write; a review panel checks for accuracy, ambiguity and bias.
- Build an item bank larger than any single exam form, so you can rotate questions and retire compromised ones.
- Pilot items before they count, and review their statistical performance. Questions that everyone gets right, or that strong candidates get wrong, are broken regardless of how good they look.
- Set the cut score with a formal standard-setting study, not by picking a round number. This is the step most homegrown programs skip, and it is the one that matters most when a failed candidate asks how you decided seventy percent was the line.
| Seventy percent is not a standard. It is a habit. A defensible pass mark comes from a documented panel process, usually a modified Angoff or similar method, in which experts judge how a minimally competent candidate would perform on each item. Keep the panel records and the scoring logs. |
Step 4: Keep certification independent from training
This is the requirement that surprises people, and it has real commercial consequences. ISO/IEC 17024 requires certification bodies to demonstrate impartiality and manage conflicts of interest. If your organization both trains candidates and certifies them, the certifying function needs genuine separation, in governance, in decision-making and often in structure.
In practice that means the people who deliver preparation courses should not also be the people who set the cut score or decide individual certification outcomes. Many associations handle this with a separate certification board or commission with its own charter. If you plan to seek accreditation, design this in from the beginning, because retrofitting independence into an existing training business is genuinely difficult.
Step 5: Decide whether to pursue accreditation, and which kind
Accreditation is optional but changes how the market treats your credential. Two routes dominate in the United States, and they are not the same thing:
| ISO/IEC 17024 (via ANAB or similar) | NCCA (via ICE) | |
| What is accredited | The certification body | The individual certification program |
| Scope | International standard | Widely recognised in the United States |
| Requires job task analysis | Yes | Yes |
| Requires formal standard setting | Yes | Yes |
| Requires impartiality separation | Yes, explicitly | Yes |
Both demand the same underlying rigour, which is why the sequencing advice is identical: build to the standard from day one, then apply. Accreditation cannot be bolted onto a program that was designed without it.
Step 6: Deliver, secure and maintain it
A certification is a living program, not a launch. Three things need ongoing attention:
- Exam security. Item banks leak. Rotate forms, monitor for disclosure, and use proctoring proportionate to the stakes of the credential.
- Psychometric maintenance. Item performance drifts as practice changes and as questions become known. Review statistics on a cycle and retire weak or exposed items.
- Recertification. Requiring continuing education or reassessment keeps the credential meaningful and creates recurring revenue. Track credit accumulation and expiry automatically rather than by spreadsheet, because a lapsed certification you failed to notice is a credibility problem.
This is where the platform matters. Registration, eligibility checks, exam delivery, proctoring, certificate issuance, continuing education tracking and recertification reminders should live in one system, because the moment they are split across tools, the record of who is currently certified stops being reliable.
The bottom line
A certification program earns its value from process, not polish. Start with a job task analysis, write a formal scheme, build the exam against a blueprint with a properly set cut score, keep the certifying function independent from training, and maintain the item bank and recertification cycle after launch. Do those things and you have a credential employers trust, an accreditation path if you want it, and a defensible answer when someone challenges a result.
MyPass LMS runs certification programs end to end, including eligibility, exam delivery, proctoring, certificate issuance and continuing education tracking, with unlimited registered users on every plan. See the platform features or start a 15-day free trial.
Frequently asked questions
How do you build a certification program?
Start with a job task analysis to define what a competent practitioner does, write a formal certification scheme, build an exam against a blueprint derived from that analysis, set the pass mark through a standard-setting study, keep certification independent from training, and require recertification to maintain the credential.
What is a job task analysis in certification?
A structured study, often involving subject matter expert panels and a practitioner survey, that documents the tasks, knowledge and skills a role requires. It produces the exam blueprint and is the evidence that a credential is job-role based rather than course-based. Accreditation bodies require it.
How do you set a passing score for a certification exam?
Through a formal standard-setting study rather than by choosing a round number. Expert panels judge how a minimally competent candidate would perform on each item, using methods such as a modified Angoff, and the resulting cut score is documented along with panel records and scoring logs.
What is ISO/IEC 17024?
The international standard for bodies that certify people. It covers impartiality and independence, certification scheme development, eligibility, exam design and administration, the certification decision, and ongoing maintenance including recertification. Accreditation to it is granted to the certification body.
What is the difference between ISO 17024 and NCCA accreditation?
ISO/IEC 17024 accreditation applies to the certification body as a whole, while NCCA accreditation, offered through the Institute for Credentialing Excellence, applies to an individual certification program. Both require job task analysis, formal standard setting and impartiality between training and certification.
Can we train and certify the same candidates?
Yes, but the functions need genuine separation. ISO/IEC 17024 requires certification bodies to demonstrate impartiality, so the people delivering preparation courses should not set cut scores or decide individual outcomes. Many associations use a separate certification board with its own charter.