The U.S. Digital Service and OPM conducted two rounds of pilots over the previous 20 months:

  • 10 completed pilots
  • 17 agencies
  • 22 offices
  • 2842 applicants
  • 419 qualified
  • 124 selections


Initial pilots (spring 2019)

What we set out to accomplish

Agencies can drastically improve hiring outcomes using current delegated examining rules and regulations by incorporating Subject Matter Experts into the process.

In spring 2019, the team tested the SME-QA process with two agencies, focusing on improving the quality of applicants appearing on certificates. Our final report and case study review the results and lessons learned.

Initial pilot results

Subsequent pilots

What we set out to accomplish

Later pilots sought to discover if the SME-QA process could flex to different assessment types and sped up through better tools to make resume reviews and interviews easier to manage.

In 2020, the team refined the process with pilots at 15 agencies, including a government-wide pilot. Additional pilots are currently underway.

Single agency pilot results

Gov-wide pilot results

Assessment and Selection Summary


Product Manager - GS-13 - 2210 series
8 Vacancies
238 Applicants
12 Passed assessments (5%)
10 Selections
6 Hired
Data Scientist - GS-12/13 - 2210 series
9 Vacancies
202 Applicants
19 Passed assessments (9%)
14 Selections
11 Hired


Customer Experience Strategist - GS-13/14/15 - 0301 series
15 Vacancies
844 Applicants
44 Passed assessments (5%)
26 Selections
18 Hired


IT Specialist - GS-13 - 2210 series
7 Vacancies
224 Applicants
25 Passed assessments (11%)
13 Selections
13 Hired


Personnel Security Specialist - GS-13 - 0800 series
4 Vacancies
112 Applicants
8 Passed assessments (7%)
4 Selections
4 Hired
IT Specialist - GS-13 - 2210 series
4 Vacancies
102 Applicants
2 Passed assessments (1%)
1 Selections
1 Hired


UX Designer - GS-15 - 301 series
3 Vacancies
146 Applicants
8 Passed assessments (5%)
4 Selections
4 Hired


IT Specialist - GS-13 - 2210 series
10 Vacancies
164 Applicants
36 Passed assessments (21%)
7 Selections
7 Hired


Grants Management Specialist - GS-12 - 1109 series
8 Vacancies
115 Applicants
27 Passed assessments (23%)
8 Selections
8 Hired
Foreign Affairs Officer - GS-13 - 0130 series
20+ Vacancies
695 Applicants
238 Passed assessments (34%)
37 Selections
37 Hired

Time to Hire Improvements

Initial pilots

The initial pilots focused on improving the quality of certificates, rather than reducing the time it takes an agency to hire applicants. However, to gather baseline information to inform future hiring actions that employ the SME-QA process, the team tracked time to hire data for the initial pilots.

HHS was the first to pilot the SME-QA process, and took 3.5 months from posting the job announcement to creating the certificates. Learning quickly from HHS, the team iterated on the SME-QA process during the DOI pilot by concurrently scheduling and conducting the first and second interviews and simplifying assessment ratings to remove the less decisive “borderline” score. These rapid iterations helped DOI complete the same hiring phases as HHS in only two months.

USDS played the role of project manager, which kept the SME-QA process moving and reduced the time to hire. As project managers, the USDS team enforced the structure and completion of the job analysis workshop, which created both a shared understanding of the SME-QA process among all participants and also built the necessary momentum to post the job announcement one week later.

When people got a cert in the past they had to hunker down for a long ugly slog. This cert came-in and was ready to go. Literally [we] can choose today.

— Hiring Manager at HHS

Diagram comparing time to hire statistics for 4 agencies who piloted SME-QA

Second round

The second round of pilots added a Resume Assessment Tool that allowed SMEs to conduct resume reviews, and for HR Specialists to audit those reviews, much faster than before. Also, written assessment reviews were used at CMS and EPA in lieu of a first round interview.