Balaji Alwar and Isaiah Baldissera, Research Fellows, Harvard Kennedy School
Rachel Lipson, Project Director, Harvard Kennedy School
--
This is the third blog post in the Project on Workforce's Skillbase series. Check out our first two blog posts on the Skillbase journey here and here.
--
Skillbase is an online learning resource aggregator for job seekers and an initiative of the Project on Workforce at Harvard University. Across the U.S., staff at public job centers, training providers, and community-based organizations use Skillbase to help job seekers connect to work opportunities and build foundational, transferable skills for their careers.
Our team at Harvard built Skillbase as a public good. Every resource that we’ve curated and featured on the platform is free for everyone. Given the Project on Workforce's research and education mission, the Skillbase team is committed to sharing out our learnings and findings. Since launching the Skillbase initiative in Spring 2020, we've gathered a significant amount of interesting data points -- conducting A/B tests, tracking usage metrics, collecting qualitative responses from user interviews. This blog post summarizes some of our key learnings from the past year.
To date, the landscape of research on designing and scaling digital offerings that truly meet worker and learner needs is relatively nascent. We hope our experiences can help advance the development and deployment of accessible and evidence-based technology in the workforce development field.
Learnings about Job Seekers
Who are the core Skillbase users? Two types of users access Skillbase regularly i) Learners and job seekers seeking to upskill themselves and ii) Staff members of Skillbase partner organizations.
Through aggregate data collection on the platform as well as user research interviews, we observed four distinctive user personas using Skillbase:
Learners seeking certifications to translate their prior experiences to high-growth jobs. For example, a non-traditional student with a publishing background in print seeking credentials in Graphic design to help communicate his/her technical skillsets.
Learners looking to upskill to find better job prospects in their current occupation. For example, a communications professional with a family working part-time in the not-for-profit sector wants to acquire persuasive writing credentials to stand out and signal their value to prospective employers.
Learners aiming to improve their English communication skills. For example, an immigrant who is trained in another country and proficient in Spanish looking to improve their interviewing skills.
Staff looking for online tools to better serve learners in a virtual environment. For example, a coach at an American Job Center is working with a client who wants to improve their resume and cover letter writing skills and uses Skillbase to recommend specific pathways/resources.
Through research, we know that the population of US job seekers has a wide range of educational as well as racial, ethnic, linguistic, and national backgrounds. On the Skillbase site, our usage data reflects a similarly diverse group of users. Skillbase users are: predominantly female (60%); a wide variety of ages (18-65); spread across the US (see map below); and primarily desktop users (~84%).
How did we drive users to Skillbase? To share the word about Skillbase to job seekers, we leveraged new and existing partnerships with career centers and non-profit organizations in Massachusetts and across the country. We ran a series of focus groups and user interviews to learn from staff of American Jobs Centers (AJCs) in Massachusetts (MassHire), as well as community-based organizations, and to get feedback on the first version of Skillbase. We then made changes to the platform based on our deeper understanding of their workflow. From here, our partners agreed to add links from their websites to Skillbase, conduct social media outreach, as well as forward our emails directly to their job seekers. A large share of Skillbase traffic came through these partners' efforts; providing early evidence for our hypothesis that intermediaries can be an effective way to reach (and provide complementary human support) to job seekers.
High demand for basic skills training. Skillbase promotes foundational learning for jobs in three core pathways: English as a Second Language, Digital Skills, and Career & Job Search. In analyzing our site's data, we see that demand for the three content areas is somewhat evenly split, with Digital Skills leading in popularity. Additional early findings on content include:
Almost half of the learners in our diagnostic survey data said they were looking to get more comfortable with technology.
More than a quarter of users highlighted that they were looking for skills relevant to job search.
Through site usage, learners showed the highest demand for basic skills across all three pathways. We observed the highest traction from resources that help 1) learners choose the right career, 2) learn the basics of English communication, and 3) gain experience using basic digital tools.
The most popular resources from our partners included USA Learns, LinkedIn Learning, Accenture Skills2Succeed Academy, GCF Global, Open Classrooms, Google Applied Digital, Duolingo.
Through their activity on the site, Skillbase learners also expressed preferences for resources that 1) offered certifications, 2) were mobile-friendly, and 3) did not require sign ups or logins.
Some learners and staff are skeptical that online training can really be free. Despite consistent messaging of the 'free' element of Skillbase across our website and email outreach, some users were still confused about Skillbase's costs (or lack thereof). One reason may have been the curation of third-party resource providers which host both paid and free content, such as Coursera. Clear and consistent communication emphasizing that Skillbase was a free resource became a key part of our outreach strategy during our live workshops.
Findings on Workforce System Staff
Supporting staff members helped Skillbase gain traction. Usage of Skillbase has always been higher during working hours on weekdays. One hypothesis for why is the critical role staff play in nudging learners to engage with Skillbase. We used surveys and user research to understand the efficacy of our partner outreach efforts and understand better how staff use the platform. The chart below summarizes some of the ways that staff users engage with Skillbase. The largest share utilizes Skillbase in onboarding and introductory coaching sessions. Others highlight the free resource through general outreach.
Staff members need to know the “why” and “how” of the tool: In interviews, we learned that staff members regularly get bombarded with a lot of different tech tools. Therefore, any new tool introduced, like Skillbase, needs to:
Clearly identify and communicate its core value proposition for staff.
Demonstrate effective ways it can be included as part of the daily workflow.
Communicating how the tool is different and solves a staff member’s problem on the job is important. We conducted interactive workshops to communicate the value of Skillbase with them and continue to utilize user research to discover creative ways to staff users to integrate Skillbase as part of their workflow.
Collateral and supplemental materials are useful for encouraging staff engagement. Coaches, teachers, and support staff work with job seekers who are demographically, cognitively, and technologically diverse. Many of these professionals are acclimated to receiving formal training resources when using any new technology tool. The Skillbase team created user guides and videos to explain the product workflow and engage staff during onboarding workshops. In some cases, staff even created their own training materials contextualizing the information for their own organizations.
Who are the staff members Skillbase serves? To complement our learning and research efforts, the Project on Workforce and Skillbase teams organized the first-ever Skillbase Community of Practice (CoP) in the Winter of 2021. The CoP was a series of professional development sessions for frontline staff working with job seekers from across non-profits, state agencies, job centers, and community colleges. The free CoP workshops served as a way to introduce Skilllbase to a broader network of professionals. In addition, though, we wanted to provide a forum for staff in the field across states and organizations to exchange their own best practices about operating in a virtual environment during COVID-19. The chart below shows the makeup of the CoP cohort of 180+ staff members.
Product Insights
User experience matters for staff and learners. We revamped the overall design of Skillbase in early 2021 with a focus on simplifying the user experience and giving the users more information about the curated resources. This led to an increase in the number of returning users (~13% to ~23%).
Defining and tracking metrics helped us test our assumptions. Tools such as Google Analytics and Hotjar helped us to identify how users are engaging with Skillbase. For example, after launching the redesigned site, we found that the Library page had 5x more clicks than any other feature or page, and so we allocated more development time to improving this part of Skillbase.
Tracking metrics helped us find anomalies in user behavior. All Internet Explorer (IE) users had a high bounce rate in the range of 90%. Some of the key features of Skillbase were not supported in the latest version of IE which led to a poor user experience. Almost 5% of our target users used Internet Explorer (IE) and as a result we did an outreach to recommend browsers such as Safari, Chrome, Edge, and Firefox instead of IE to those users.
Integration of third-party libraries helped with the rapid testing of hypotheses. One of our hypotheses was that a significant number of learners needed to translate the site into Spanish for easy navigation. To test this assumption, we quickly utilized a 3rd party service to integrate a translation widget in Skillbase. We validated this hypothesis as we observed that a substantial percentage of users started using the translation widget.
Focus on reducing friction points over adding features. Given the audience for Skillbase, simplicity was one of our primary design principles. One of our hypotheses was that learners want personalized recommendations, and therefore, would be willing to login or signup to track their progress and usage over time. However, another competing hypothesis was that having signups/logins as part of the mandatory workflow will create friction for learners. We enabled this feature as part of the workflow to test this hypothesis. The login/signup option received very low traction from learners. We learned that login/sign up was a challenge and decided to not make this workflow mandatory.
Guiding Questions for the Future
As we move forward, these are the questions we ask ourselves as we plot out our next steps using the Build-Measure-Learn feedback loop:
How might we better integrate Skillbase as part of a staff member's workflow and outreach with job seekers?
How do we reach more learners and staff? What outreach strategies can we adopt that can help us create impact at a significant scale?
How might we design and develop the next version of Skillbase, keeping in mind users of diverse abilities/disabilities and diverse racial, ethnic, linguistic, and national backgrounds?
Skillbase will continue to learn, grow, and iterate as we continue to learn about our users. We are inspired by the mission of this work and excited about the opportunity to work with outstanding partners across the country. We're happy to answer questions related to our work to help others building similar products. To get in touch, please reach out to the Harvard Project on Workforce team. You can also check out the Skillbase Partners Page to learn more about upcoming opportunities.
Comentários