Written by: Alexander Jettel & Jeff Wikstrom

At Nuvalence, we are excited about OpenAI’s recent ChatGPT developments and the possibilities that their new plugin architecture enables. Recently, we had the opportunity to check out the new ChatGPT Plugins capability for ourselves (you can read our overview here).

In this post, you will learn about our experience developing a ChatGPT plugin end-to-end, including how to create and test a plugin, as well as some best practices we recommend and some pitfalls you may encounter. We’ll use Alexander Jettel’s very own plugin, VideoInsights, as an example. VideoInsights is rapidly gaining popularity as a way to use ChatGPT to access transcripts of videos published on platforms like YouTube or Dailymotion.

Key Components of a Plugin

To develop an OpenAI ChatGPT plugin, there are four essential components to consider:

  1. A Manifest
  2. An OpenAI specification
  3. The API itself
  4. Authentication

We’ll examine each of these elements in detail and provide insights into their implementation.

The Manifest

The manifest file contains general information about the plugin you are developing. This JSON file includes, but is not limited to: the name of the plugin, a description, the authentication set up, and information about where the API is hosted. 

Let’s have a look at an example manifest below and dive into the most important configuration values.

{
 "schema_version": "1.0.0",
 "name_for_human": "Video Plugin",
 "name_for_model": "video_content",
 "description_for_human": "Plugin for getting the content and other information of a Video Links.",
 "description_for_model": "Plugin for getting the content and other information of a Video Links.",
 "auth": {
   "type": "none"
 },
 "api": {
   "type": "openapi",
   "url": "https://nuvalence.ai/.well-known/openapi.yaml",
   "is_user_authenticated": false
 },
 "logo_url": "https://nuvalence.de/logo.png",
 "contact_email": "contact@nuvalence.com",
 "legal_info_url": "https://www.nuvalence.com/legal"
}

NOTE: These are the required parameters that must be set when creating a plugin, which we’ll delve into more closely. There are additional, optional parameters beyond what we cover here, which can be found in OpenAI’s documentation.

The schema_version parameter is mainly for internal versioning purposes, and can be set to any string. However, we recommend adhering to industry standards of semantic versioning for consistency.

The name_for_human, description_for_human, and logo_url parameters dictate the text and image displayed on the OpenAI UI when the plugin is installed and used. These parameters are crucial for creating a clear and concise representation of your plugin. TIP: Plugins that include copyrighted names or trademarks will not be accepted by OpenAI.

The name_for_model and description_for_model parameters are used by OpenAI’s GPT model to determine whether your plugin must be called to answer the current ChatGPT prompt. These parameters should be phrased accordingly, and accurately describe the plugin’s purpose. If these are not described properly, OpenAI would not call your plugin for the task.

The auth and type parameters are used for authentication and its configuration. These crucial aspects of a ChatGPT plugin are complex, which we’ll cover in an upcoming post. For now, we have set the authentication parameter to “none”, meaning there is no authentication required after the installation of the plugin, and end-users can utilize the plugin without logging in.

The api parameter has three components, only one of which you need to alter. The url component points to the location of your API’s OpenAI YAML specification (which brings us to the next important file for developing an OpenAI plugin).

The OpenAPI Specification

The OpenAPI YAML specification contains the layout of your API. If you decide to use a cloud provider like Google Cloud, AWS, or Azure to host your API endpoints, you can utilize their API Gateway products to automatically export this specification file. 

Whenever OpenAI determines that your API may be required, it downloads this file to determine the correct endpoint to call. This means you can update the specification separately from the manifest, and add, remove, or update new endpoints.

Let’s continue with our video API example. Below, you’ll find a breakdown of the segments of a complete OpenAPI specification, outlining an API that provides a video transcript: the headers, the paths, and the components.

The Headers 

openapi: "3.0.1"
info:
 title: "openai-video-api"
 description: "OpenAI Video API Gateway"
 version: "2023-04-01T07:13:27Z"
servers:
- url: "https://videoapi.nuvalence.com"

First, we define a general section with a free-form title, description, and version. TIP: Adopt semantic versioning to ensure you and your developers can maintain a comprehensive and organized record of releases and version history.

The “servers” entry points to the API URL at which the requests can be made, which can be different from the domain at which you will host your OpenAI manifest and OpenAPI specification.

The Paths 

The example below is a standard implementation of the OpenAPI Paths object specification. Here, we define two endpoints of the API: GetFullTranscript and GetVideoSummary. The implementation details behind the API endpoints are up to you, but names should clearly describe the functionality.

To ensure that OpenAI comprehends and leverages the appropriate API calls, a concise summary entry must be provided for each endpoint and method. Moreover, it is important to adopt a descriptive naming convention for the paths, operations, and parameter names. The ChatGPT plugin relies on these naming conventions and descriptions to accurately identify the right API endpoint to invoke, and the requisite parameters to include. Hence, it is crucial to pay close attention to these details to optimize the utilization of OpenAI’s capabilities.

paths:
 /getFullTranscript:
   get:
     operationId: "GetFullTranscriptOperation"
     summary: "Gets the full transcript of a video"
     parameters:
     - name: "video-url"
       in: "query"
       required: true
       schema:
         type: "string"
     - name: "timestamped"
       in: "query"
       schema:
         type: "string"
     responses:
       "200":
         description: "200 response"
         headers:
           Content-Type:
             schema:
               type: "string"
         content:
           application/json:
             schema:
               $ref: "#/components/schemas/FullTranscriptResponse"
 /getVideoSummary:
   get:
     operationId: "GetVideoSummaryOperation"
     summary: "Gets a summary of a video"
     parameters:
     - name: "video-url"
       in: "query"
       required: true
       schema:
         type: "string"
     responses:
       "200":
         description: "200 response"
         headers:
           Content-Type:
             schema:
               type: "string"
         content:
           application/json:
             schema:
               $ref: "#/components/schemas/VideoSummaryResponse"

The Components

This leads us to the components part of the OpenAI specification. The example below shows the implementation of the components section for a ChatGPT plugin. Here, we define two reusable schemas: VideoSummaryResponse and FullTranscriptResponse. These are used as the return schemas for the API endpoints. 

To optimize this section, it is essential to ensure that these components are accurately defined, and their naming conventions and descriptions are consistent with the rest of the API. The ChatGPT plugin relies on these components to ensure that the appropriate data is passed and retrieved accurately. 

components:
 schemas:
   VideoSummaryResponse:
     title: "VideoSummaryResponse"
     type: "object"
     properties:
       metadata:
         type: "object"
         properties:
           is_age_restricted:
             type: "boolean"
             description: "Whether the video is age-restricted or not"
           keywords:
             type: "array"
             description: "Keywords related to the video"
             items:
               type: "string"
           length_in_seconds:
             type: "string"
             description: "Length of the video in seconds"
           number_of_views:
             type: "string"
             description: "Number of views of the video"
           video_title:
             type: "string"
             description: "Title of the video"
         description: "Metadata of the video"
       status:
         type: "integer"
         description: "Status code of response"
       summary:
         type: "string"
         description: "The summary of the video"
       video-url:
         type: "string"
         description: "Video url of the transcript"
   FullTranscriptResponse:
     title: "FullTranscriptResponse"
     type: "object"
     properties:
       metadata:
         type: "object"
         properties:
           is_age_restricted:
             type: "boolean"
             description: "Whether the video is age-restricted or not"
           keywords:
             type: "array"
             description: "Keywords related to the video"
             items:
               type: "string"
           length_in_seconds:
             type: "string"
             description: "Length of the video in seconds"
           number_of_views:
             type: "string"
             description: "Number of views of the video"
           video_title:
             type: "string"
             description: "Title of the video"
         description: "Metadata of the video"
       status:
         type: "integer"
         description: "Status code of response"
       timestamped:
         type: "boolean"
         description: "Whether the transcript includes timestamps"
       transcript:
         type: "string"
         description: "The full transcript of the video"
       video-id:
         type: "string"
         description: "Video url of the transcript"

The API

The API should be hosted on a cloud provider like AWS, Google Cloud, or Azure to enable fast, seamless scaling and ensure maximum uptime. Cloud providers offer a wide range of services that can enhance the performance, security, and reliability of your API. Self-hosted hardware can be challenging to scale up and down rapidly, making cloud hosting a better option for  OpenAI plugins. 

Deployment

Once you’ve established your manifest, OpenAPI specification, and your API, it’s time to test your integration with OpenAI.

To deploy your integration, simply access the ChatGPT Plugin store and click on the “Install an unverified plugin” option located at the bottom of the popup.

Screenshot of ChatGPT plugin store

OpenAI will prompt you to provide the domain of the plugin you wish to install. There are two ways to proceed: you can either host the plugin locally and use a local proxy for installation, or you can upload your manifest and API specification to a domain.

Hosting on localhost.  If you choose to host the plugin on localhost, you will need to set up a local proxy as described in the OpenAI specification.

Hosting on a domain. Opting to host the plugin on a domain requires uploading your manifest and OpenAPI specification file to your domain’s .well-known subdirectory. This will make them accessible at domain.com/.well-known/ai-plugin.json and domain.com/.well-known/openapi.yaml.

At this stage, you can input the URL of either your localhost proxy or your domain to complete the installation process.

Screenshot of UI showing input box for URL of either your localhost proxy or domain.

Testing

To test the plugin, activate it in the ChatGPT UI and execute a prompt that should trigger your API. As long as your endpoints are correctly named, your summaries are descriptive, and your variable names are clear, ChatGPT will handle the rest automatically.

Screenshot of successful plugin activation.

You can confirm that your plugin was successfully invoked by checking for a message above the response from ChatGPT. If you need more information about the call and response, simply click on the small arrow next to the message.

Screenshot of confirmation that plugin activated

At this stage, it’s a good idea to follow best practices for testing. Aside from edge cases and load testing, you can also consider testing for interoperability with other plugins or examining how the plugin performs under different network conditions.

Best Practices

Now that we’ve reviewed the essential components, let’s explore some best practices to keep in mind when developing a plugin.

  • HTTPS. When releasing a plugin on the store, https: is required on the domain that hosts your API and your files. Without it, you are relegated to an unapproved status, which limits maximum installations to 15 at the same time.
  • Query response time/caching. Take care to optimize your API response time by precomputing or caching answers. After receiving a response from your API, ChatGPT needs to process the information before sending the final response to the end user. This introduces additional latency, so an optimization strategy can help improve the user experience.
  • API versioning. To develop a production API while in active use, it’s advisable to version your endpoints. This approach allows you to simply update the OpenAPI specification file with new paths as soon as you’re ready to release changes to ChatGPT.
  • Observability and error handling. Adhere to observability best practices for your API to ensure that it’s functioning correctly, performing well, and being utilized appropriately. If your API encounters errors or is unavailable, ensure that the proper error messages are communicated to ChatGPT so it can respond accordingly instead of timing out, which would diminish the user experience.

Conclusion

Developing a plugin for ChatGPT presents an exciting opportunity to extend and customize the capabilities of this transformative AI model. Throughout this post, we’ve unraveled the process and essential components involved in creating a plugin, from the Manifest and OpenAPI specification to the API, Authentication, Deployment, and Testing. We also touched upon key best practices that will help maintain optimal performance and user experience.

One point that stands out throughout the process is the simplicity of creating a plugin. With a clear understanding of the requirements and steps, even developers who are new to ChatGPT can confidently craft a plugin and begin extending the model’s abilities. The comprehensive documentation provided by OpenAI further eases the process, making the creation of a plugin more accessible and straightforward.

Another important aspect of ChatGPT plugin capability is authentication. Stay tuned for the next post in the series to learn more about what we learned about authentication methods!


Interested in exploring ChatGPT plugins? Now that the ChatGPT Plugin store is more widely available in its beta release, many exciting plugins have been added. If you’re a ChatGPT Plus user, make sure to check out the store for VideoInsights and more (check out this LinkedIn post from The Neuron AI newsletter co-founder Pete Huang for more on VideoInsights and other influential plugins).

Nuvalence can help you unlock the possibilities of AI to propel your business forward.

LET’S TALK