Thank you for getting in touch, we'll get back to you soon!

Do you have some insights to share with the product community?

GET IN TOUCH.

AI Won’t Replace PMs. But It Will Divide Them

Written by
Unmukt Raizada Unmukt Raizada
Founder & CPO @ TrustEvals.ai
Published
Share

Two distinct parts are emerging in the world of product management with AI. This distinction matters a lot. When you understand where you fall and where you want to go (from a career trajectory path), you can align accordingly.

There is the AI-enabled PM , AI PM, and AI-infrastructure PM (which we are clubbing into an AI PM for broader understanding)

Unmukt pic 1
AI Enabled PM

Background

This frankly will be the bare minimum, and if you think of it as Maslow's hierarchy, then this will be the bare minimum that is expected from every PM going forward. Here, PMs will get comfortable with using various tools to enhance and improve their efficiency as well as increase their coverage.

Unmukt pic 2

The old world: PMs were stretched across an impossible list—customer calls, PRDs, sprint planning, QA, GTM alignment, documentation, stakeholder management—with barely any time left for actual product thinking. The common refrain was "I need three more people just to do my job."

The new world: AI tools collapse into a tighter loop. What used to require a product analyst, a QA specialist, and hours of documentation can now be compressed into focused workflows augmented by AI. The PM who masters these tools doesn't just work faster—they work at a fundamentally different level, spending more time on strategy and less on task execution. The top 10% of PMs that understand how to use AI will be 10x more productive with fewer resources. What makes it even more exciting is that their contribution will not be limited to just product anymore, as AI enables them to go in and push smaller design and functional changes with the right levels of governance.

To that note, here is a take that might not be well received by a larger percentage of PMs globally. AI code generators are now already at the level where minor bug fixes, as well as design-oriented feedback and experience-oriented feedback, can be driven directly by the PMs and pushed for code review. With the right governance in place to ensure that code reviews are done thoroughly, the AI enabled PM can finally do what the traditional PM couldn't, and ensure a crisp user experience that best reflects their version of quality.

What does a typical day look like?

  1. Key Focus Area: A large set of PMs narrow down their focus in the first half of the day to a key focus area that they had already pre-planned. It typically takes up at least 25-30% of your day. This could either be:
  2. A new deeper product implementation writing what in the old world used to be a PRD, but in the new world will be a combination of PRD + working prototype through lovable replit, base44 or emergent.
  3. Market research on a new product segment that either the customers are pushing the company towards or the company sees as an opportunity to increase its impact.
  4. Getting functional and technical alignment from key stakeholders, which includes but might not be limited to GTM, sales, CS, engineering, and broader leadership.

  1. Analytics: Explore the analytics that are involved with the usage of key features and product. It will also combine customer sentiment that has come in the previous day and look to articulate what are the key bits that need a review. This includes using tools like Amplitude, Mixpanel or Enterpret for analytics as well as FullView, log rocket and the likes for actually seeing some customer sessions in a sample mode. Especially at scale, it becomes harder to see all customer sessions. But using Amplitude mix panel or Interpretable to see activity from the previous day allows you to narrow down your focus on the specific type of customers you want to see live sessions of. This typically lasts for about 30-60 minutes.
  2. Live customer session: A PM needs to be very, very close to their customers, and a daily call for 30 minutes will do more for unlocking creativity than most other tasks.A key feature of this live customer session is the discoverability aspect, so one needs to go in curious and not hung up on their assumptions.Typically, having an Excel tracker for this is critical as it ensures that there is discipline in terms of where you are double-clicking and what opportunities you are identifying.If you use Granola and are able to create folders within it for specific types of calls and customers, it allows you to use their API to dump all the notes into specific projects and then use it even more effectively. This tends to be about 30-60 minutes.
  3. Daily syncs across multiple teams: Sitting across marketing, sales, customer success, Engineering product support, possibly technical support, and ensuring that you're pulling all the strings that need to be pulled for your product to be its best version.
  4. Prototype feedback from customer: Speak to the customers that care deeply about specific product features, and use prototyping tools to walk them through how they'll likely be implemented. Once the customer comes back with feedback, you'll be able to iterate quickly through it. If a customer truly cares about a feature, they will be willing to hop on a call with you, which allows you to narrow down the turnaround time and give engineering a well-laid-out spec vs. a half-baked version of what you have interpreted from the requirements that a customer may have disclosed to you.
  5. Rapid testing of new features / enhancements: A PM is responsible for the overall quality of the product and therefore spends about 15-30 minutes per day in testing out the new features and announcements to ensure that they are up to the quality bar set for the organization.
  6. Push out minor fixes associated with user experience or feature drops that you feel confident in using Claude Code/ Cursor.

Typical AI Tools used

  1. Note taking/ Meeting Recorder: Granola, Gmeet / Zoom/ Teams for video recordings/screen capture
  2. Voice to text: WisprFlow
  3. Product Analytics: Enterpret, Amplitude, Mixpanel
  4. Customer Sessions: FullView, LogRocket
  5. Synthesizing information: Your Frontier model of choice (ChatGPT, Claude, Gemini). Ensure you use projects, skills, and their connectors across Notion, Docs, Folders, Slack/Teams etc
  6. Minor Fixes/Coding requirements: Claude Code, Cursor
  7. Prototyping: Claude Code, Replit, Emergent, Base 44, Lovable, Figma Make
  8. Ticketing: Linear
  9. Documentation: GDocs , Notion, Word

AI PM

Background

BackgroundAI PMs are those that are building AI features and actually getting them delivered to their end user. Now examples of these could be a chatbot or an internal chat GPT use case or summarizing call transcripts. An AI experience PM is one that must be able to articulate what the problem is, how the solutioning is going to work at a high level and a deeper design level, and one that has a very strong hold on LLM evaluations to ensure that the last mile delivery is meeting the quality and confidence that goes with building probabilistic systems. They also need to have a deeper understanding of a lot of the UX challenges that go with building AI products and experience more broadly. This also ties into the ability to create edge cases and figure out what are the guardrails that will be needed. An AI PM is one that has a good grasp of how transformer-led architecture actually influences LLM behavior.

AI PMs are definitely more technical in that they understand when to use an LLM and when a traditional machine learning approach or even a more pure play deterministic workflow might do the job.

Unmukt pic 3

This decision-making framework is core to the AI PM's value. Getting it wrong means over-engineering simple problems or under-serving complex ones.



Within AI PMs is a spectrum of technical depth that needs to be considered.

  1. Integration & Experience: Focused on prompt design, UX for probabilistic outputs, guardrails, and cross-functional work with design and engineering. Your work breaks when the prompt changes or user expectations shift.
  2. Model & Data: Focused on fine-tuning, training data pipelines, labeling workflows at scale, and evaluation infrastructure. Your work breaks when the training data changes or model performance degrades.
  3. AI Infrastructure: Focused on the foundational systems—model serving, training pipelines, platform tooling—that other AI teams build upon. Your work breaks when the underlying platform can't scale or support new capabilities.

Unmukt pic 4

What does a typical day look like?

  1. Evals: Begin by reviewing the overnight eval results and validating the auto-grader scores, flagged responses as well as edge cases that have been called out. Wherever needed, update the golden dataset with the results from the production traces. This can take anywhere from 15 minutes to an hour if done daily.
  2. Triage edge cases from human raters to confirm that the grading has happened at the right granularity. Categorize all the failure modes that have popped up and work through their solutioning from prompt engineering as well as guardrails. In parallel, look through what types of customers are impacted and ensure that your next steps are calibrated with regards to the seriousness of the failure modes.
  3. Work through the prompt adjustments, guard rails, updated evals in the new pipeline that will be created, and help engineering understand what needs to be done here and articulate the cost associated with it.
  4. Think through the user experience of today and the user experience of tomorrow. While the user experience of today will likely still be in app with the advancements being made across the GPT SDK, there is a real need to be API-first. Such that LLMs can use your MCP or similar protocol to interact with the app. Ensure that you are designing your product accordingly and spend time in looking at how customers are starting to use your AI product. This will be a key part of the day.
  5. Create launch criteria docs with updated test data sets for the feature iterations that you are currently working through. Understand what are the benchmarks that you need for these features to hit the sweet spot. Also, ensure that your production traces are segmented basis type of customer and type of product so that you can make quick judgment calls for the results to figure out how deep you wanna go.
  6. Shared Activities with PMs that use AI:
    1. Live customer session
    2. Daily syncs across multiple teams
    3. Prototype feedback from customer
    4. Rapid testing of new features / enhancements
    5. Minor fixes/ UI & UX improvements

Typical AI Tools used

Apart from the tools mentioned above for the AI-enabled PMS, the below tools will be used:

  1. LLM Playgrounds: Use your Frontier model to test prompts, tweak parameters, find edge cases etc
  2. Prompt Engineering: LangSmith, PromptLayer, DSPy (+GEPA)
  3. Auto Graders: Custom Scripts, RAGAS, DeepEval, 3rd party consultants (TrustEvals.ai)
  4. Human Rating : Scale.ai, 3rd party consultants (TrustEvals.ai)
  5. Observability: Arize, Braintrust
  6. Feedback collection: Pipeline that takes user feedback (qualitative and quantitative)
  7. Guardrails: Moderation + Perspective + Guardrails APIs, Custom rule engines + regex filters.

How to move from an AI-enabled PM to an AI PM.

Start with the technical foundation

You don't need to become an ML engineer, but you do need to understand how LLMs actually work as systems. Understand transformer architecture at a conceptual level, why models hallucinate, what temperature and context windows actually do, and when an LLM is required versus when traditional ML or deterministic logic. Resources like Andrej Karpathy's lectures or the Anthropic research blog are accessible entry points.

Get hands-on with evals before you're asked to

The single biggest skill gap between these roles is evaluation design. AI-enabled PMs judge success with traditional product metrics. AI PMs need to define what "good" means for probabilistic outputs—and that's genuinely hard (also the reason we at trustevals.ai make a living out of it).

Start experimenting: build a small eval set for an AI feature you use, try auto-grading with a frontier model, learn what failure modes look like. Tools like LangSmith or even simple spreadsheet-based golden datasets will teach you the mechanics.

Find or create a bridging project

The cleanest path is owning an AI feature within your current scope. Volunteer to lead a summarization feature, a chatbot implementation, or an AI-assisted workflow. This gives you legitimate exposure to prompt engineering, guardrail design, and the UX challenges of non-deterministic outputs—without requiring a full role change.

Build relationships with ML and data science teams

Sit in on model reviews, and ask engineers to walk you through how they're thinking about training data or latency trade-offs.

Reframe your narrative

Once you have done the above, position yourself as someone who has already shipped AI-adjacent work, understands the unique challenges, and is ready to go deeper. The transition is less about a single leap and more about accumulating credibility through progressively technical projects.

Share

ABOUT THE AUTHOR

Unmukt Raizada

Unmukt Raizada
Founder & CPO @ TrustEvals.ai

Unmukt Raizada is the Founder & CEO of TrustEvals.ai. He has spent 16 years building at the intersection of data and AI - 12 years on data and machine learning at Goldman Sachs, JPMorgan Chase, Airtel Digital, and Wipro, followed by 4 years hands-on with generative AI, including deploying AI agents at enterprise scale at Thena.ai. As co founder in a Lightspeed backed company, Unmukt built the product that is used by companies like Cloudflare, Clickhouse, Vercel, Braze and Amplitude. At TrustEvals, he helps enterprises ensure their AI agents deliver the value they were built to create.

If your AI agents are deployed but you're not sure they're delivering the outcomes you invested in, TrustEvals can help. Learn more at trustevals.ai or reach Unmukt directly at unmukt@trustevals.ai.

Related articles

Insights and thoughts from leading product people.

Looking to build your product team or find the perfect role? Let’s chat