GPT-3.5 is really cheap (prompt and completion = $0.002 / 1K tokens), but GPT-4 is around 20 times more expensive (prompt = $0.03 / 1K tokens + completion = $0.06 / 1K tokens).

But the benefit from using the API is that you can change the model on the fly, so you chat with 3.5 until you notice that it's not responding properly and, with all the history you have (probably stored in your database), you can send a bigger request with a probably better response once with GPT-4 as the selected model.

I really wish the interface on chat.openai.org would allow me to switch between models in the same conversation in order to 1) not use up your quota of GPT-4 interactions per 3 hours as quickly and 2) not strain the backend unnecessarily when you know that starting a conversation with GPT-3.5 is efficient enough until you notice that you better switch models.

OpenAI already has this implemented: When you use up your quota of GPT-4 chats, it offers you to drop down into GPT-3.5 in that same conversation.

Sure, but GPT-4 through the UI costs $20 per month, which is a lot of api calls.

Isn’t it 10 per hour?

25 / 3 hrs

How is it that cheap?! I ran three queries on langchain yesterday with two ConstitionalPrompts and it cost $0.22 - made me realize deploying my project for cheap could be expensive quick.

GPT3.5 Turbo pricing is 10k tokens or ~7500 words for $0.02. Though note that every API request includes the entire chat context and charges for input & output tokens. https://openai.com/pricing

You need to check which model you are using, also... LangChain runs through the model several times with increased token count on each successive call.

Yeah I assumed it would be doing several times but still more expensive than OP mentioned. I think the issue is I'm using davinci-003

Yeah, davinci-003 is gonna be gpt3, which is more expensive than 3.5.

One more anecdote: I've been running a half dozen gpt3.5 IRC bots for a few weeks and their total cost was less than a dollar. A few hours of playing around with LangChain on gpt3 cost me almost $4 before I realized I needed to switch to 3.5, though even then it still uses a ton of tokens every chain.

Thanks, I'll do that later

I’d love to see a comparison of the average cost to use this with the OpenAI API versus subscribing to chat-gpt plus.

Maybe I’ll have to try this for a month and see if it end up costing more than $20. Thanks for creating it!

Wow! Is it really that cheap? GPT4 is much more expensive, I imagine?

GPT-4 is decently more expensive -- I personally really like & use the therapist character a lot. In this scenario the session would cost me less than $1 which is still much cheaper than any therapist I've used previously :)

What is your setup?