Results

The U.S. Digital Service and OPM have conducted two rounds of pilots and subsequent SMEQA hiring actions since 2019.

  • 17 completed actions
  • 42 agencies
  • 6450 applicants
  • 933 qualified
  • 406 selections
  • 274 accepted offers

Pilots

Initial pilots (spring 2019)

Proving the concept

Agencies can drastically improve hiring outcomes using current delegated examining rules and regulations by incorporating Subject Matter Experts into the process.

In spring 2019, the team tested the SME-QA process with two agencies, focusing on improving the quality of applicants appearing on certificates. Our final report and case study review the results and lessons learned.

Initial pilot results

Subsequent pilots

Expanding possibilities

Later pilots sought to discover if the SME-QA process could flex to different assessment types and sped up through better tools to make resume reviews and interviews easier to manage.

In 2020 and 2021, the team refined the process with pilots at further agencies, including cross-government pilots. SMEQA actions have continued in 2022.

Single agency pilot results

Cross agency CX pilot results

Assessment and Selection Summary

CMS: MULTIPLE OFFICES

Product Manager - GS-13 - 2210 series
8 Vacancies
238 Applicants
12 Passed assessments (5%)
10 Selections
6 Accepted offers
Data Scientist - GS-12/13 - 2210 series
9 Vacancies
202 Applicants
19 Passed assessments (9%)
14 Selections
11 Accepted offers
Product Manager - GS-13 - 2210 series
9 Vacancies
215 Applicants
11 Passed assessments (5%)
10 Selections
9 Accepted offers

CX: MULTIPLE OFFICES

Customer Experience Strategist - GS-13/14/15 - 0301 series
15 Vacancies
844 Applicants
44 Passed assessments (5%)
26 Selections
19 Accepted offers
Customer Experience Strategist - GS-13 - 0301 series
15 Vacancies
553 Applicants
47 Passed assessments (8%)
31 Selections
12 Accepted offers

DATA: MULTIPLE OFFICES

Data Scientist - GS-13/14 - 0343 series
62 Vacancies
513 Applicants
107 Passed assessments (20%)
105 Selections
50 Accepted offers

DHS: OIT

IT Specialist (INFOSEC) - GS-12/13 - 2210 series
20 Vacancies
170 Applicants
26 Passed assessments (15%)
18 Selections
13 Accepted offers

DOI: NATIONAL PARK SERVICE

IT Specialist - GS-13 - 2210 series
7 Vacancies
224 Applicants
25 Passed assessments (11%)
13 Selections
13 Accepted offers
IT Specialist - GS-13 - 2210 series
8 Vacancies
200 Applicants
18 Passed assessments (9%)
8 Selections
6 Accepted offers

EPA: OFFICE OF MISSION SUPPORT

Personnel Security Specialist - GS-13 - 0800 series
4 Vacancies
112 Applicants
8 Passed assessments (7%)
4 Selections
4 Accepted offers
IT Specialist - GS-13 - 2210 series
4 Vacancies
102 Applicants
2 Passed assessments (1%)
1 Selections
1 Accepted offers

GRANTS: MULTIPLE OFFICES

Grants Management Specialist - GS-11 - 1109 series
15 Vacancies
749 Applicants
102 Passed assessments (13%)
41 Selections
13 Accepted offers

GSA: MULTIPLE OFFICES

UX Designer - GS-15 - 301 series
3 Vacancies
146 Applicants
8 Passed assessments (5%)
4 Selections
4 Accepted offers
Contract Specialist - GS-09 - 1102 series
50 Vacancies
1208 Applicants
203 Passed assessments (16%)
30 Selections
25 Accepted offers

HHS: CIO AND CTO

IT Specialist - GS-13 - 2210 series
10 Vacancies
164 Applicants
36 Passed assessments (21%)
7 Selections
7 Accepted offers

STATE: MULTIPLE OFFICES

Grants Management Specialist - GS-12 - 1109 series
8 Vacancies
115 Applicants
27 Passed assessments (23%)
8 Selections
8 Accepted offers
Foreign Affairs Officer - GS-13 - 0130 series
20+ Vacancies
695 Applicants
238 Passed assessments (34%)
76 Selections
73 Accepted offers

Time to Hire Improvements

First round of pilots

Round one pilots focused on improving the quality of certificates, rather than reducing the time it takes an agency to hire applicants. However, to gather baseline information to inform future hiring actions that employ the SME-QA process, the team tracked time to hire data for the initial pilots.

HHS was the first to pilot the SME-QA process, and took 3.5 months from posting the job announcement to creating the certificates. Learning quickly from HHS, the team iterated on the SME-QA process during the DOI pilot by concurrently scheduling and conducting the first and second interviews and simplifying assessment ratings to remove the less decisive “borderline” score. These rapid iterations helped DOI complete the same hiring phases as HHS in only two months.

USDS played the role of project manager, which kept the SME-QA process moving and reduced the time to hire. As project managers, the USDS team enforced the structure and completion of the job analysis workshop, which created both a shared understanding of the SME-QA process among all participants and also built the necessary momentum to post the job announcement one week later.

When people got a cert in the past, they had to hunker down for a long ugly slog. This cert came-in and was ready to go. Literally [we] can choose today.

— HHS Hiring Manager

Second round of pilots

Round two pilots added a Resume Review Tool that allowed SMEs to conduct resume reviews, and for HR Specialists to audit those reviews, much faster than before, along with a self-scheduling tool for interviews. Written assessment reviews were used at CMS and EPA in lieu of a first round interview. The results were striking: 5.5 weeks from announcement to certificate, and both EPA and CMS were able to make immediate selections.

Diagram comparing time to hire statistics for 4 agencies who piloted SME-QA

Continued SMEQA actions

In 2022, agencies continued to conduct hiring actions using SMEQA practices. CMS's fall 2021 hiring action took less than five business weeks from announcement to certificate. Other agencies are rolling out and developing hiring practices to more fully utilize SMEs in a variety of ways.