Real-Time AI Voice with Azure OpenAI GPT-4o & Node.js (TypeScript)

Microsoft recently added support for real-time audio output via the gpt-4o-mini-realtime-preview model. In this tutorial, you'll learn how to use it from a Node.js TypeScript app, send a prompt, and play back the AI-generated speech in real-time.

🛠️ Prerequisites

• Node.js v18+ (I'm using v22.15.0)

• TypeScript installed globally

• Azure OpenAI resource with gpt-4o-mini-realtime-preview deployed

• .env file with:

  AZURE_OPENAI_ENDPOINT=<Endpoint for your Azure OpenAI resource>
  AZURE_OPENAI_API_KEY=<Key to your API>
  AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4o-mini-realtime-preview
  OPENAI_API_VERSION=<API version>
  

---

📦 Setup

1. Project Initialization

mkdir azure-gpt4o-audio && cd azure-gpt4o-audio
npm init -y
npm pkg set type=module
npm install openai @azure/identity dotenv speaker

Your package.json file might look something like this

2. Environment Config

Create a .env file:

AZURE_OPENAI_ENDPOINT=<Endpoint for your Azure OpenAI resource>
AZURE_OPENAI_API_KEY=<Key to your API>
AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4o-mini-realtime-preview
OPENAI_API_VERSION=<API version>

---

🎧 TypeScript Code

---

▶️ Run Your App

You can run the application using the npm run start shorthand that I've got configured in my package.json. It transpiles the Typescript into a JavaScript file and runs the output JS.

Your speaker will now play back the AI’s voice response in real time 🎧.

📌 Final Thoughts

Once you've built this application, read through the API reference and try to build on this by adding more features (maybe a 'Stop' button to stop the audio response?).