How to Build an AI Career Navigator

AI can help professionals navigate their careers with personalized training and mentorship opportunities. I break it down from a product perspective on how to go from 0 to 1 into making it a reality.

I’m a big fan of continuous learning - not only is it fun, but it’s also critical for a product career in tech and digital roles. In my fair share of courses and webinars, there were always the same kind of question being asked during Q&A:

(For learning a new skillset for work): This is great! But what about my work situation?

(For career development): What if my career goals are XYZ and my background is ABC?

“What about Me?”

That’s the question.

Everyone’s trying to navigate their own career map, and nobody’s journey is the same. Nobody’s challenges, background, or existing skillsets are the same.

Yet courses are designed to teach the most common fundamentals - then it’s up to you to go out there and start applying them.

To be fair, catering to the widest possible audience is strategic of the education platforms and course creators. Meanwhile, online topical communities on LinkedIn and social media platforms can narrow your exploration into more relevant areas - you can find groups on just about any subject, but the quality of engagement can vary.

Even within these smaller communities, I still hear the question, “What about Me?”

And what about you?

What about your specific challenges and context?

What does navigating your career map look like?

The Idea

What if we used AI (LLMs) to help you navigate your career?

Imagine if there was a search engine that understood your unique career challenges and goals, and found the most relevant courses for your career development.

Knowing the career paths and aspirations of many users*, the model could match you to mentors who have walked the same road you're trying to embark on.

Educators and thought leaders can identify unmet needs and learning goals of career aspirers, use that feedback to create targeted, personalized courses and content, carving out their niche and reaching their target audiences more effectively**.

*LinkedIn data could be integrated via partnerships, LinkedIn could develop this itself, or another web platform could curate the career aspirations and background of its users.

**If users allowed for their personal career goals and background to be shared with education platforms. Privacy should be maintained with anonymized data, and unless enrolled in a course, a user’s personal info is never shared.

The good news is that recommendation engines like those on LinkedIn and Instagram already exist. Employing LLMs wouldn’t necessarily mean overhauling all that work, but be more of an integration with existing recommendation AI to improve its ability to understand users’ needs and preferences.

Why is this exciting now?

Because now we have LLMs like ChatGPT, Gemini and Claude.

Traditional NLP models required you to use keywords they were programmed to recognize, and were able to do simple pattern matching. But NLPs will miss a lot of context and deeper meaning.

On the other hand, LLMs are much more capable of understanding nuances and context, and are capable of analyzing tons of unstructured data from diverse sources (that’s how they’re trained). LLMs would able to absorb inputs from users, course descriptions, mentor profiles, and then generate personalized recommendations.

Let’s take my own career as an example:

I was a non-STEM major who explored career paths in teaching and journalism before getting my MBA. At Rice, I fell in love with startups, became the External Lead for Rice University’s startup incubator OwlSpark, and learned through absorption. Combining my love for startups and business learnings from my MBA, I wanted to be a tech Product Manager - except I didn’t have any experience in tech, product management, and I wasn’t even an engineer.

Luckily, networking led to my first PM role at HPE. Later, I taught myself to code, went to Reliant (NRG) as the Digital Product Manager for the Innovation team, and went on to lead product for 2 brands, working on a diverse portfolio of products and programs.

I’ve always been driven to make a positive impact - I believe product management is about creating scalable solutions to real problems. I’m exploring ways for AI to solve problems and improve how we build products, and enabling people and organizations to innovate better. In the long-term, my sights are on product leadership roles where I can set strategy and org design to enable teams to make good things.

If you fed my career narrative into a traditional natural language processing (NLP) model, and asked it to develop a career plan, it won’t be able to generate responses. If you put it into a search engine, it’s going to give me a list of Rice MBA and startup links.

However, if you fed my narrative to an LLM model, it can give you this (courtesy of Claude):

Short-Term Goals (1-2 years):

• Pursue certifications or courses in AI product management (e.g., AI PM Specialization)
• Take on AI-focused projects within your current role or on the side.
• Network and learn from AI product leaders in the industry

Mid-Term Goals (3-5 years):

• Transition into an AI-focused product role at your company

Long-Term Vision (5+ years):

• Explore opportunities in executive product leadership roles

Much better! But not enough - which courses? Which AI product leaders have the most relevant expertise? What if my background is non-STEM?

Without additional data (course descriptions, career backgrounds), out of the box LLMs aren’t able to provide more detailed guidance.

But deployed in a setting where that data exists and can be fed to the model, then we’ve got some real potential.

If you’re interested in learning more about NLPs and LLMs, my notes from Coursera’s AI Product Management Specialization covers this topic in more detail.

Case Study with Reforge

Reforge is an online education platform where classes are built by real product and growth leaders. All the courses are based on practical, real-world experiences of actually doing the work. I LOVE Reforge and the community - it’s been the best career building experience I’ve had.

I recently had the pleasure of being interviewed by Reforge about their courses. I had been debating whether to take Yue Zhao’s course on Breaking Barriers: From Manager to Executive (I’m paying out of pocket for my learning).

Having seen Yue’s amazing webinars, I was already strongly considering signing up, but was still hesitating. What got me excited and registered was reading the ‘Who this course is for’ section:

I felt called out by the highlighted sections - this was so relevant to me! It’s like Yue had pulled insights from the depths of my career and put them down on paper.

Inspired, I fired off a follow up email to Reforge and outlined the potential I saw for online learning. Here’s the condensed version (modified w/an LLM for conciseness):

Reforge could leverage AI to automatically match users with highly relevant courses. During account creation, users would provide their goals, challenges, and expectations (this could be manual input at first, but can also evolve into an interactive AI-powered "informational interview" using speech-to-text transcription. The AI would summarize key points, confirm with the user, and store their profile in a database). This database would be matched against course descriptions scraped from the "Who this Course is For" sections, enabling personalized course recommendations. The most relevant courses would be surfaced on the Home page after login.

Additionally, for users open to mentorship, the platform could pair them with appropriate mentees. Mentors and mentees will be matched based on their backgrounds, career goals, and the relevance of mentors’ roles to the mentees’ next career steps.

For course creators, Reforge could analyze the user profile data to identify common challenges, skills gaps, or niche problems that existing courses don't fully address. The AI could surface these insights to course creators, allowing them to spot opportunities for developing new, targeted course content. Users can modify their goals and preferences anytime from their dashboard. Reminders for quarterly check-ins would allow them to reflect on their goals and mentorship availability. The Reforge AI chatbot, which can already be used to find relevant course content to user queries, could further augment recommendations by inferring needs based on user prompts (with permission).

This personalized, AI-driven approach could significantly enhance the user experience and boost enrollments by curating and surfacing highly relevant course content and career development opportunities.

Here’s what it could look like:

Key Questions Before Building

I’m making a lot of assumptions. Is this a better solution to enough people to be viable as a product?

As my MBA Strategy professor would say, “Where’s the data?”

We need low-risk, low cost tests to validate our hypotheses. Here’s how I’d go about it:

Proof of Concept

If we fed a LLM with the right type of data, will it improve responses on career guidance? It’s relatively easy to do this with LLM tools, so you can even start here to validate if there’s something there.

• Create a custom GPT on ChatGPT, or simply put your prompt in a new conversation with Claude and Gemini, upload CSV files of real course descriptions and synthesized user career data, and test for yourself.
• Tweak prompting, compare different LLMs

ID which model metrics to track

• The easiest thing to do is looking at the Chatbot Arena Leadership Board and see if you agree with the rankings of LLMs.

• Of the LLMs that have a user-friendly interface that allows you to upload docs and prompt, the most advanced AI models would be ChatGPT 4o from OpenAI (available free with limited access), Gemini Advanced from Google (free 2-month trial), and Claude 3 Opus from Anthropic (not free, but you can try the free model Sonnet first and see if it’s worth paying for). Pricing information accurate as of June 7, 2024 - check links for the latest.

Later on, you would likely need custom metrics not already on the leaderboard - e.g. F1 score for relevance, diversity in course recommendations, ROUGE for summarization, AUC-ROC for classification quality, etc.

• This could mean running a custom Python script and connecting to APIs of models - luckily, Claude and ChatGPT are excellent coders for these experimental tasks. You can easily follow instructions to setup your own Python environment, run tests on select models, and debug any issues while having a chatbot explain how the code works in layman’s terms.
• I’ve done this to create a Reddit content scraper with TextBlob sentiment analysis over the weekend, and I’m not a Python developer.

Size the market/demand

• Research the market size for online learning platforms
• Find existing market research in the domain to see if there is a real pain point around getting personalized career development help.

(Later) Create your own survey to validate the size of the target market.

• Surveys are something you should do after interviews, so you know which questions to ask and have an idea of who you’re targeting to size. Doing a survey right off the bat is too open ended to be useful - take the time to do interviews and get thoughts from real people first before you start trying to abstract and quantify.

Competitive landscape

What solutions already exist?

• LinkedIn seems to be a natural fit to deploy this, since users’ certifications, roles, and backgrounds already exist. However there’s risks of fake profiles creating garbage data.

What’s the current status quo most people are using to navigate their careers?

Interviews with potential users

Interviews are great, I love them for discovering things I didn’t even think to explore. Even if you only speak to a handful of people, you can learn so much, and it helps you scope your quantitative testing efforts.

• If you have a 1-pg concept to share (a landing page that describes your idea), you can get very quick feedback.

Ask about the competition - What are they currently using to solve for the problem, how satisfied are they, and what frustrations do they still experience?

• Online and DIY research can supplement, but nothing replaces that empathy you develop for people when they’re sharing their pain points. If you’re not solving a problem for yourself (where you already feel that pain), then you need to feel that pain from others.

Interviews will also help you ID target users for prototype testing - everyone needs career guidance, but who seem to be the ones with the most need, and are the least satisfied with current solutions?

Survey to narrow down the applicable market size

This should be based on your current hypothesis of who your real target users are, informed by the interviews.

• Note - you may NOT have the market size figured out until after you’ve already launched something. People are creative at adapting tools in ways you didn’t even imagine a use case for - launch small, watch for ‘peaky’ user behaviors that look abnormal from what you expect, and figure out why.
• At this point, at least have a sense that there’s enough of a demand out there to bother beginning.

Dogfood testing - before testing with real users

From the interviews, you may discover that you need to update your proof of concept.

Setup a test rubric for human evaluation - what results are you looking for that signals a ‘good response’?

• This is another reason why interviews are so important - you may learn about signals people are looking for.
• This should also feed into the evaluation questions for your rapid prototyping tests with real users.

Rapid prototypes and testing w/real users

Create interactive prototypes (Figma or Keynote), design interview guides and screener surveys, recruit on platforms like UserTesting, conduct and record sessions, upload transcripts into LLMs for analysis and review recordings of key insights.

• I always end up rewatching recordings because testers tend to NOT want to write down their thoughts. I created a Voiceflow prototype around this need and put it into a blog post.
• Evaluate pricing sentiment - Van Westendorp Analysis and Max Diff surveys can help narrow down what price ranges people are most willing to pay, and for which features.

Estimating the ROI

What would API costs look like?

• How much user input text are you handling, and how many tokens are being passed to an expensive LLM API? Simple fixes could include limiting the number of characters users can input. About 0.7 tokens per word is the rough estimate for a quick, back of the hand calculation.
• To reduce API costs - which tasks make sense to use an NLP with vs an open ended LLM? Can we route manage certain queries to appropriate LLMs with cheaper APIs? Can we use smaller LLM models like BERT for understanding user inputs?

After factoring in the Return (size of applicable market, validated improved product value proposition, and pricing through testing) vs the Investment (API, development and maintenance costs) - is there a positive ROI?

Product people - how would you improve this approach? Leave me a comment!

The Bigger Picture

There’s potential for a whole new web platform to aggregate career narratives and background data from LinkedIn, and course content data from education platforms like Reforge, Udacity, Coursera, Maven, to provide a comprehensive career development service for personalized learning, career guidance, and mentorship. It can also provide analytics to course creators to develop new niche courses targeting smaller markets with unmet needs.

One thing’s for sure - AI is having us explore how to future proof our careers. Unfortunately, the demand for skills often mismatches the supply in the labor market, especially for those from already underserved communities.

It may be critically important, now more than ever, to develop personalized career training and mentorship opportunities that are accessible and relevant for today’s fast paced demands.

Wherever you are in your career journey, it’s always worth starting now.

Thank you! Let me know your thoughts and feedback in the comments.