Round One Pilots

In January 2019, the U.S. Digital Service (USDS) and Office of Personnel Management (OPM) partnered to test Subject Matter Expert Qualification Assessments (SME-QA), a process where SMEs work with HR specialists to create qualification criteria based on the SMEs' determination of what is required to be successful in the job from day one.

These SMEs then use these criteria to conduct resume review and phone interview assessments. The USDS/OPM Hiring Pilot Team hypothesized that with SMEs completing these assessments before HR specialists considered an applicant qualified and eligible for veterans’ preference, hiring managers would receive higher quality certificates and make more selections.

The SME-QA pilots have shown that agencies can drastically improve hiring outcomes using current delegated examining rules and regulations. However, the pilots presented challenges to both HR specialists and SMEs given the time commitment and lack of technology to automate what are currently highly manual tasks. Nonetheless, interviews with all participants during and after the pilots have demonstrated that they are glad to have participated and feel the SME-QA process was worthwhile. Both DOI and HHS intend to apply the process to future hiring actions.

We were so used to a year [instead of two months]. When we told some people we hired this many people…they said 'How many years did it take?'

— Hiring Manager at DOI

By the numbers

The most important metric the team tracked was the number of selections hiring managers made from the certificates, indicating the quality of the certificates. Because the SME-QA process gradually eliminates applicants who SMEs do not find qualified, hiring managers received a shorter list of qualified applicants. In both pilots, fewer applicants were deemed qualified but more applicants were selected.

Health and Human Services

IT Specialist - GS-13 - 2210 series
10 Vacancies
164 Applicants
36 Passed assessments (21%)
7 Selections
7 Hired

Department of Interior

IT Specialist - GS-13 - 2210 series
7 Vacancies
224 Applicants
25 Passed assessments (11%)
13 Selections
13 Hired

The team also hypothesized that if presented with a list of truly qualified applicants, hiring managers would make selections faster than they would during a typical delegated examination hiring process. Baseline data shows that, on average, hiring managers across government take 37 days to make a selection after receiving certificates, representing the longest phase of the hiring process.2 In the SME-QA process, hiring managers made their first selections between 11 and 16 days after receiving the certificates.

We got the results that you all promised. We have a list of 36 people who can do the job.

— Hiring Manager at HHS

Cost benefit analysis

We averaged the time required of both HR specialists and SMEs from both pilot agencies that participated in the SME-QA process, which helped the team calculate the average cost for each selection made off of a certificate.

If the pilot agencies that used the SME-QA process made six selections off a certificate, the cost per hire would become equal to the industry average, and any selections over six would reduce the cost per hire considerably. If DOI and HHS had ended up making offers to all qualified applicants, their cost per hire would have dropped to between $700 - $1000.

Role FTA Hours per FTE
SME 8 57.5
HR 2 115

Assumes 150-200 applicants

Total hires Cost per hire
5 $6,692
10 $3,346
15 $2,231
20 $1,673
25 $1,338
30 $1,115
Society for HR Management $4,425
Benchmark for Federal Agencies (External Hires) $10,561