To Build or not to Build?

AI continues to revolutionize tech-enabled industries and product management teams face the critical challenge of effectively adopting and integrating AI into their products, workflows, and processes. As a product leader, you have likely encountered the ‘Build vs Buy’ decision for your products and features at some point. Now throw AI technologies into the mix, and it further complicates this decision-making process.

Let’s explore a very brief generalized framework to guide you through this decision-making process and lay out key considerations on selecting the right AI model or technology, if you choose to buy.

• First, we’ll look at some key Decision Parameters and their related questions.

• Next, I’ll introduce a simple 2x2 decision matrix to guide the decision making process.

• Finally, we’ll look at key considerations for AI Technology selection.

Before we delve deeper, there are a few important points to consider:

• It's important to note that there is no one-size-fits-all framework. The interpretation and application of the framework is likely to be subjective based on the industry/ vertical/ sector your company operates in, company stage, the maturity of the Tech Stack (e.g. AI-First vs AI-Led/ Enhanced vs Traditional), talent readiness, and the existing product organization culture.

• In this article, “AI solution” serves as a catch-all term and my intent is to encompass various aspects like:

  • Foundational multimodal AI models/ APIs (think, Commercial LLMs from OpenAI, Anthropic, Google and open source LLMs like LLama3 from Meta)

  • Off-the-shelf turnkey platform solutions (e.g. Intercom’s Customer Service AI platform, Cloud based AI services, Fraud detection systems, Assistants)

  • AI development frameworks (e.g. Langchain)

  • Specialized AI Tech (e.g. NLP services, Computer Vision services) and more! (not just GenAI)

  • Proof of Concept (PoC) for a specific product use-case

  • Note: Going into the nuances in detail will be out of scope for this article.

• The assumption here is that the ‘Build vs Buy’ decision would typically factor into the early stages of AI adoption and consequently, the product teams should have already conducted a thorough assessment to identify potential AI use cases and opportunities, in alignment with business strategy. Thought starters you can use:

  • What new opportunities do the new technologies open up?

  • What new experiences do these new technologies enable?

  • What can your engineers solve today that you couldn't solve before?

Decision Parameters

When considering whether to buy or build an AI solution for specific use-cases, you want to start by evaluating the following key parameters and questions:

Business Alignment

Guiding considerations

1. Is there an actual problem/ user need that can be solved effectively if AI were to be leveraged? (Don’t do AI just for the sake of doing AI!)

2. How well does the proposed solution align with your company's core competencies and strategic goals?

3. Will building or buying provide a competitive advantage/ moat in your market (Both, short-term and long-term)?

Key questions:

• Does this AI solution directly contribute to our unique value proposition?

• How critical is this AI capability to our long-term strategy?

Time-to-Market

Guiding considerations

1. How quickly do you need to implement the AI solution?

2. Will building in-house significantly delay your product launch or feature release?

Key questions:

• What is our target timeline for implementing this AI capability?

• How would the build vs. buy decision impact our ability to meet market demands?

Strategic Value

Guiding considerations

1. Will building in-house create intellectual property that could be valuable in the future?

2. Does buying allow you to leverage external expertise?

3. What’s the organization’s long-term innovation strategy?

Key questions:

• How might developing this AI capability in-house contribute to our long-term innovation strategy?

• What strategic partnerships or knowledge transfer opportunities come with buying a solution?

Technical Feasibility

Guiding considerations

1. Does your team have the necessary expertise in-house to build and maintain an AI solution?

2. Are there existing solutions that meet your technical requirements?

3. What level of customization will be required for the use-cases identified?

Key questions:

• What specific AI capabilities do we need, and how complex are they?

• (Specialized expertise) Do we have relevant in-house talent to develop and evolve this AI technology?

Resourcing

Guiding considerations

1. What human and computational resources are required for building vs. buying?

2. How will this decision impact your team's capacity for other projects?

Key questions:

• Can we allocate sufficient resources to build and maintain an in-house solution?

• How would buying a solution free up our team for other strategic initiatives?

Cost Analysis

Guiding considerations

1. What are the upfront and long-term costs associated with building vs. buying?

2. Have you considered hidden costs such as maintenance, updates, and scaling?

Key questions:

• What is the total cost of ownership for building vs. buying over a 3-5 year period?

• How do these costs align with our budget and expected ROI?

Data Control

Guiding considerations

1. How important is data ownership and privacy for your use case?

2. Are there regulatory considerations that impact your ability to share organizational and user data with third-party vendors?

3. Is vendor lock-in and future switching costs a big concern?

Key questions:

• What level of data control and privacy do we require for this AI solution?

• How would building vs. buying impact our ability to comply with data regulations in our industry?

2x2 Decision Matrix: Build vs Buy

To simplify the decision-making process, here’s a 2x2 matrix that considers two critical factors: Strategic Value and Implementation Capability (of the product organization). This matrix can help guide your decision in most situations:

Let’s explore each quadrant:

1. Buy Standard Solution (Low Strategic Value, Low Implementation Capability):
   When the AI solution isn't critical to your core business and you lack the internal capabilities to build it, buying a standard solution is often the best choice.

2. Build Selectively (Low Strategic Value, High Implementation Capability):
   If you have the capability to build but the solution isn't highly strategic, consider building selectively. This might be worthwhile if you can create a solution that's significantly better than available options.

3. Strategic Imperative (Build) (High Strategic Value, High Implementation Capability):
   When the AI solution is crucial to your business strategy and you have the capability to build it, this is typically a clear case for building in-house.

4. Capability Gap (Partner) (High Strategic Value, Low Implementation Capability):
   If the solution is strategically important but you lack the capability to build it, consider partnering with a vendor or consultancy to develop a custom solution while building internal capabilities.

AI Model/ Technology Selection Framework (If Buying)

If you decide to buy an AI solution, use the following considerations to select the right model or technology for a production use-case, to ensure that your AI investments deliver maximum value to the organization.

Problem Definition

1. Clearly articulate the specific problem you're trying to solve with AI.

2. Identify the desired outcomes and success metrics.

Vendor Evaluation

1. Research potential vendors' track records, support offerings (SLA? Ongoing consultation?), and product roadmaps.

2. Consider the vendor's expertise in your specific industry. (Review case studies and success stories relevant to your use-case)

3. Assess the vendor's data handling practices and security measures. (e.g. GDPR)

4. Understand how they protect your data during training and inference processes.

Data Considerations

1. Assess the quantity, quality, and type of data required to fine-tune the AI model (e.g. GPT4) to perform optimally for your specific product and business use-cases.

2. Evaluate data integration capabilities with your existing systems and knowledge bases.

Performance Requirements

1. Define the accuracy, speed, and scalability needs for your use case. (You don’t necessarily always need the best LLM model in the market for your use case)

2. Consider the model's ability to handle edge cases and evolve over time.

3. Use relevant LLMOps tools to prototype and compare different models’ performance over sample data.

Scalability and Integration

1. Evaluate how well the solution can scale with your business and product growth.

2. Assess the ease of integration with your current tech stack and workflows.

Wrapping up

The decision to ‘Build vs Buy’ AI solutions is complex and multifaceted. Remember that this isn't always an all-or-nothing choice – a hybrid approach, combining bought solutions with custom-built in-house components may sometimes be the optimal path. Stay informed about emerging technologies and be prepared to adapt your strategy to maintain a competitive edge in this market.

Happy building (with) AI!