Home Assistant ChatGPT Integration for Natural Language Control

In this article, I will show you how to set up the Home Assistant ChatGPT-4o and 3.5-Turbo model in your Home Assistant to control your smart home using natural language. My goal is to make conversations with Home Assistant feel like talking to a real person without spending a lot of money. For example, you can say, “Okay Nabu, make me a coffee if it’s after 10 a.m.”. First, I’ll show you the end result and how everything works. Then, I’ll explain the setup process and the costs involved.

End Result (Home Assistant ChatGPT Integration)

There are several ways to control Home Assistant using OpenAI ChatGPT and the Assist functionality.

• Text Commands: This is the slowest method, but visually straightforward. You type your command, and Home Assistant responds. For example, I typed, “Make me a coffee if it is after 10:00 a.m.” and it responded, “Coffee maker is now on. Enjoy your coffee.”

• Voice Commands with Devices:
  • ESP32-S3-Box-3 Device: This costs around $60-$70. I have a tutorial on configuring this device – Switching from Alexa to the ESP32-S3-Box-3: My Experience
  • M5 Atom Echo: This device costs around $15 and also supports voice commands.I also have a tutorial here – Setting up M5 ATOM Echo for Home Assistant wake word control | Easy Guide!

These devices use a wake word to activate and execute commands. For example, saying, “Okay Nabu, turn on kitchen lights if it is after 9 p.m.” will start the lights maker if the condition is met.

Are you a visual person?

If you prefer to watch video instead of reading, no problem here is my video tutorial

Setting Up OpenAI GPT in Home Assistant

Here’s a step-by-step guide to setting up OpenAI GPT in Home Assistant:

Step 1: Create an OpenAI Account

1. Go to the OpenAI website and create an account.
2. Log in and enable billing by adding your credit card. You can set usage limits to control costs. I personally added $5 and disabled auto-recharge and I didn’t set any usage limits

Step 2: Generate a Secret Key

1. Create a secret key from the OpenAI dashboard.
2. Copy the secret key somewhere safe for just in a minute use in Home Assistant.

Step 3: Add OpenAI Integration in Home Assistant

1. In Home Assistant, go to the Integrations section and add a new integration.
2. Search for “OpenAI” and paste your secret key.

Step 4: Configure the Conversation Agent

1. Edit your existing pipeline or create a new one as described here – link
2. Select “ChatGPT” as your conversation agent in your pipeline.
3. You can customize the agent’s behavior by modifying the prompt. For example, you can make it act like a specific character, such as “You are Harry Potter from Harry Potter series. Be funny.”

Step 5: Set Up Local Add-ons

1. Faster-Whisper: For speech-to-text processing.
2. Piper: For text-to-speech.
3. OpenWakeWord: To activate the device with a wake word.

NOTE! Above steps are for local speech processing. If you are Home Assistant Cloud user, you can use a cloud pipeline as part of your subscription.

Step 6: Test and Debug

1. Test various commands to ensure everything is working.
2. Use the pipeline debug feature to see what commands and responses are being processed.

Controlling Access and Security

You can control which devices or entities are accessible via voice commands and respectively Home Assistant Assist and ChatGPT:

1. Click on the device settings (cog wheel button).
2. Go to Voice Assistant settings.
3. Enable or disable control as needed.

For example, if you don’t want the assistant to control your front door locks, you can disable access for that specific device.

Cost Breakdown

Using OpenAI GPT with Home Assistant can be cost-effective only if you are using older and not so smart models (GPT-3.5-Turbo. Here’s a breakdown of my expenses:

• GPT-4o: 76 cents for 108 requests.
• GPT-3.5-Turbo: 1 cent for 14 requests.

Overall, latest and greatest GPT-4o model is expensive if you are using it a lot on daily basis. The Older GPT-3.5-Turbo is a lot more affordable, but is dumber, yet still effective.

Conclusion

Using the OpenAI GPT model with Home Assistant makes controlling your smart home with natural language easy and fun. With a few simple steps, you can set up a system that understands and responds to your commands, making your smart home more interactive and user-friendly.

The other AI option for Home Assistant (for now) is Google Generative AI which is 14 times cheaper than ChatGPT. If you have any questions or want a detailed tutorial, let me know in the comments!

Thank you for ChatGPT-ing with me I’m Kiril and I will see you in the next one.