ChatGPT can be a valuable productivity improvement tool for product managers when we understand what it excels at and keep its limitations in mind.
We recently published an infographic that outlines the Eight steps for successful product management. Using these steps, we decided to evaluate ChatGPT. Here is how it performed:
- Know your purpose: ChatGPT isn’t a mind-reader
ChatGPT offers a limited ability to assist in understanding and defining the purpose of your product, because the purpose is borne out of your company’s unique history, strengths and vision. Your expertise plays a crucial role in developing a purpose that encompasses and conveys your product’s reason for being and the specific value it offers to your stakeholders and users.
- Map personas: great as a starting point
ChatGPT is a good starting point for persona development. For instance, when given the prompt, “I'm building a product management software that will be used by associate product managers, product managers, group product managers, and chief product officers. Create a user persona for these four users,” ChatGPT provides their estimated age, goals and motivations, challenges they face, and desired features and benefits. Obviously, you would have to adjust it for nuances and subtleties only you as a product visionary would know.
- Define activities and features: a useful research tool
ChatGPT is a useful tool for creating a quick inventory of features for competing products. However, when you ask it to imagine new features for a new distinctive product you are envisioning, it recommends creating features that are already built into existing products. It can’t create something new, it can only draw from existing information. It’s not Steve Jobs, who imagined an iPhone by combining existing technologies. Or Mike Beedle who gave birth to the Agile and Scrum Manifesto by observing the limitations in applicability of Six Sigma and Lean Manufacturing to Software Development.
- Prioritize work and options: not capable yet
ChatGPT lacks the necessary context and higher-level understanding to make intelligent choices when it comes to prioritization. Knowing your team’s strengths and weaknesses, skillset, and the prioritizing of features based on their business value, requires a more nuanced understanding that’s beyond the capabilities of ChatGPT.
- Align activities to goals and KPIs: very limited
While ChatGPT can discover general KPIs and metrics used in various industries, its capabilities when it comes to aligning your activities with goals and key performance indicators (KPIs) are very limited – you are on your own when it comes to effectively completing this step.
- Plan release and roadmap: not a helpful tool
ChatGPT doesn't possess the contextual understanding required to make more than a minimal contribution to the tasks that go into release planning and roadmap development. These are dependent on competitive analysis, buying trends and market conditions, and your team’s talents and skills.
- Track product health and metrics: average for suggestions
ChatGPT can offer suggestions for tracking product health and metrics, however, it requires considerable effort and critical thinking to interpret and implement these suggestions effectively.
- Communicate with teams and stakeholders: good timesaver
Communicating to align various teams is one of the most time-consuming aspects of a product manager’s life. If you craft your prompts with detailed input, ChatGPT can assist with communications tasks, such as:
- writing marketing requirements documents (MRDs)
- preparing product requirements documents (PRDs)
- developing upcoming product release notes for the marketing team
- summarizing user feedback, new feature requests, interview scripts
- creating FAQs for customer support and marketing
As long as you humanize the robotic-sounding copy it produces – and round it out with your common sense, creativity, and contextual knowledge – ChatGPT can be a good tool that can reduce the time you spend on these tasks.
ChatGPT can clearly improve the productivity of product managers.
Mind the gaps
Using ChatGPT requires vigilance and alertness, especially because of its two glaring limitations:
ChatGPT’s retrospective view inherently conflicts with innovation
ChatGPT operates by analyzing published documents, which gives it a historic and retrospective view – it pulls from existing data and then spits out a new version of what’s been written. However, at its heart, product management is all about forward-thinking innovation and creating new value that doesn't exist yet. At this stage of development, ChatGPT can’t drive innovation.
Sopheon recently introduced the concept of InnovationOps, which aims to harmonize people, organizational culture, processes, and technologies to foster consistent innovation at scale. ChatGPT was unable to contribute to envisioning this novel concept – it kept steering the conversation toward older, agile processes like Scrum and Kanban instead.
ChatGPT sounds authoritative despite errors and mistakes
Tonally, ChatGPT's responses sound authoritative, even if they are not entirely accurate. While it seems like we don’t trust authority on the surface, Yale psychologist Stanley Milgram’s electric shock experiments conducted in the 1960s tell us otherwise. The findings revealed an alarming human tendency to obey and defer to authority even if its mandates contradict personal, moral consciences. It’s important to always approach ChatGPT's responses with a critical mindset and not blindly accept them as absolute truth.
Ultimately, ChatGPT should not be considered a substitute for human ingenuity, experience, common sense, debate, and discussion. To gain the most value from using ChatGPT in your product management process, you can use it to improve your productivity.
Since innovations in generative AI are moving at breakneck speed, stay tuned for more.