At Nuvalence, we are all in on Generative AI, and we’re especially excited about a new, limited/beta-release capability from OpenAI: ChatGPT Plugins. 

ChatGPT on its own is magic, but plugins are pure wizardry. Want access to data after September 2021? ChatGPT plugins can deliver that. Want to enable your users to tap into their data directly from the existing ChatGPT experience? ChatGPT plugins can deliver that. Want to take it a step further and allow users to take action on your application within ChatGPT? ChatGPT plugins can deliver that. 

It could be the next big app store ecosystem, and, if so, will revolutionize the way businesses think about their software development investments. It’s orders of magnitude easier to build than a native app or web application, and arguably more user-friendly.

While ChatGPT Plugins are now available in beta for premium subscribers, the ability to develop them is invite-only as of this writing. We’ve been lucky enough to check out both and are excited to share our experience and perspective so far.

How do ChatGPT plugins work?

A ChatGPT plugin is incredibly simple as a concept. It is an API and a few configuration files (Open API Definition and Plugin Manifest) deployed on a public domain. When deployed to ChatGPT, a plugin is simply a pointer to the domain where the API and those files are hosted, which the ChatGPT Plugin Model will use if toggled for use. See the diagram below. 

Exhibit 1: ChatGPT Plugin Model

Building a ChatGPT Plugin: 5 Easy Steps

We found that ChatGPT plugins are very straightforward to create; we did so in 5 steps.

TIP: This is intended as a high-level overview. For a deeper dive, check out the publicly available docs from OpenAI, explore this Github repo, and look out for our next blog post in this series!

1. Build and deploy an API that’s accessible from a public domain. We tested this using a Node.js Express API on Google Cloud’s Cloud Run.
   
2. Create and deploy a Plugin manifest + Open API definition on the same domain. We used the same Express Node.js app to serve these files.
   
3. Navigate to the Plugin Store in ChatGPT, choose the “Develop your own plugin” option, and validate that your API and configuration is set up properly for installation.

4. Install your “unverified plugin.” At this point, you can share your plugin’s URL and enable up to 15 others to test it.

5. If you want to submit your plugin for public availability, you can do so via the ChatGPT Plugin Submission bot.

As you can see, the process was pretty simple, but you might still have a few questions. I know we did:

Q: How do plugins authenticate?

A: While most apps we tested were fully public and did not include authentication, you can choose between a number of different authentication approaches from service level to user-specific OAuth. Check out the OpenAI guidance to learn more. 

Q: When responding to a prompt, how does ChatGPT know which plugin to call, the specific API to utilize, and how to send the right information?

A: First, ChatGPT leverages the description from the Plugin Manifest to decide when a plugin should be used in response to a prompt. Then, ChatGPT utilizes information in the OpenAPI definition to decide which endpoint, or set of endpoints, it needs to use to complete its task. It does this by viewing the summary value of each endpoint, which is defined in simple plain language. That’s it. There is no glue code. When it decides on an endpoint to use, it intelligently pulls out the relevant entities from the prompt and inserts them into the fields submitted to your API. When we tested this, it worked surprisingly well and with a high degree of accuracy.

Q: How do you manage the user experience based on the information being sent back to ChatGPT from your API? 

A: You don’t. You simply provide data to ChatGPT, and it decides how to utilize that response from your API when replying to the user’s prompt. In some ways, this is ideal; you don’t need to add that layer of polish to your response, and it makes utilizing existing APIs much easier. But on the other hand, it really limits your control over the user experience.

Q: How do you make changes to published plugins?

A: If you need to make changes to the API or functionality that alter the OpenAPI definition or Plugin Manifest, you need to resubmit the application for review. (Currently there is a 7-day waiting period.)

What We’d Change: 5 Opportunities for Improvement

While the beta version of ChatGPT plugins that we used is incredibly exciting, in practice, we found several improvement opportunities that will help it fulfill its potential.  

Here are 5 specific areas we’d love to see improved in future releases:

1. Lift the plugin limit constraint. Currently, ChatGPT imposes a 3-integration limit when using plugins, which forces you to select them for each use. Once you download a plugin to your account, it should always be in play while using ChatGPT.
   
2. Allow private plugin deployment. Currently, there is no way to “privately” deploy a plugin – all deployed and approved plugins are publicly available to all ChatGPT users. Offering a path to deploy a plugin and provide experiences that are specific, and private, to an organization would unlock opportunities for developers to use the capability for a wider range of use cases.
   
3. Improve the plugin user authentication experience. When a plugin requires authentication outside of ChatGPT, users are routed externally to provide credentials, and aren’t routed back to ChatGPT or given feedback on whether authentication was successful. This creates a confusing user experience and detracts from the value of the capability.
   
4. Support for calling the plugin model with selected integrations enabled using the API. This would enable developers to leverage plugins while using OpenAI API so that the output can be integrated into experiences outside of ChatGPT. For example, this could even be used to work around the 3-integration limitation mentioned above. You could first ask GPT-4 to identify which plugin was most likely to be used based on the question being asked, and then call the model with those specific plugins enabled.
   
5. Allow ChatGPT Plugin APIs to utilize the user account to make additional calls to OpenAI. Currently, when a ChatGPT user utilizes a plugin, and that plugin’s API needs to interact with OpenAI APIs (for instance, to summarize extensive video content), the API developer must use their own API key. Consequently, they bear the OpenAI usage charges. This can lead to substantial costs when dealing with high information throughput scenarios. It would be more cost-effective, and less burdensome, if plugin APIs could use the user’s account when further interacting with the API in the context of plugin usage.

Our Verdict?

After thoroughly testing the beta version of ChatGPT’s new plugin capability, we can confidently say that it’s innovative, user-friendly, and has potential to transform how organizations leverage generative AI. Even at this early stage, it demonstrates maturity, and we’re excited to see how OpenAI will refine it.

In fact, we found the capability so useful that we didn’t just stop at testing it – our Principal Engineering Manager Alexander Jettel created a pretty cool production-ready plugin called VideoInsights, which enables you to use ChatGPT to access transcripts of videos published on platforms like YouTube or Dailymotion.

Stay tuned for our next post, where we’ll provide an in-depth look at how we developed the VideoInsights plugin and share the best practices we identified along the way. And in the meantime, you can check out the live plugin at the ChatGPT Plugin store!