Round One Pilots
In January 2019, the U.S. Digital Service (USDS) and Office of Personnel Management (OPM) partnered to test Subject Matter Expert Qualification Assessments (SME-QA), a process where SMEs work with HR specialists to create qualification criteria based on the SMEs' determination of what is required to be successful in the job from day one.
These SMEs then use these criteria to conduct resume review and phone interview assessments. The USDS/OPM Hiring Pilot Team hypothesized that with SMEs completing these assessments before HR specialists considered an applicant qualified and eligible for veterans’ preference, hiring managers would receive higher quality certificates and make more selections.
The SME-QA pilots have shown that agencies can drastically improve hiring outcomes using current delegated examining rules and regulations. However, the pilots presented challenges to both HR specialists and SMEs given the time commitment and lack of technology to automate what are currently highly manual tasks. Nonetheless, interviews with all participants during and after the pilots have demonstrated that they are glad to have participated and feel the SME-QA process was worthwhile. Both DOI and HHS intend to apply the process to future hiring actions.
We were so used to a year [instead of two months]. When we told some people we hired this many people…they said 'How many years did it take?'
— Hiring Manager at DOI
By the numbers
The most important metric the team tracked was the number of selections hiring managers made from the certificates, indicating the quality of the certificates. Because the SME-QA process gradually eliminates applicants who SMEs do not find qualified, hiring managers received a shorter list of qualified applicants. In both pilots, fewer applicants were deemed qualified but more applicants were selected.
Health and Human Services
IT Specialist - GS-13 - 2210 series
Department of Interior
IT Specialist - GS-13 - 2210 series
The team also hypothesized that if presented with a list of truly qualified applicants, hiring managers would make selections faster than they would during a typical delegated examination hiring process. Baseline data shows that, on average, hiring managers across government take 37 days to make a selection after receiving certificates, representing the longest phase of the hiring process.2 In the SME-QA process, hiring managers made their first selections between 11 and 16 days after receiving the certificates.
We got the results that you all promised. We have a list of 36 people who can do the job.
— Hiring Manager at HHS
Cost benefit analysis
We averaged the time required of both HR specialists and SMEs from both pilot agencies that participated in the SME-QA process, which helped the team calculate the average cost for each selection made off of a certificate.
If the pilot agencies that used the SME-QA process made six selections off a certificate, the cost per hire would become equal to the industry average, and any selections over six would reduce the cost per hire considerably. If DOI and HHS had ended up making offers to all qualified applicants, their cost per hire would have dropped to between $700 - $1000.
| Role | FTA | Hours per FTE |
|---|---|---|
| SME | 8 | 57.5 |
| HR | 2 | 115 |
Assumes 150-200 applicants
| Total hires | Cost per hire |
|---|---|
| 5 | $6,692 |
| 10 | $3,346 |
| 15 | $2,231 |
| 20 | $1,673 |
| 25 | $1,338 |
| 30 | $1,115 |
| Society for HR Management | $4,425 |
| Benchmark for Federal Agencies (External Hires) | $10,561 |