OpenAI's First DevDay

[LAwLz]

Summary

OpenAI, the small company that less than a year ago revolutionized the entire field of computing by releasing things like ChatGPT and Dall-E, is holding their first of presumably many conferences today. 

 

What will be announced remains to be seen.

Current rumors and speculations include:

• New developer tools. 
• New models. 
• A new "team" payment plan aimed at companies. 
• A framework for building custom chatbots. 

 

Quotes

Quote

 AI

-Probably someone at OpenAI

 

 

 

The announcements:

 

Starting with some numbers.

• 2 million developers are developing using OpenAI APIs.
• 92% of Fortune 500 companies use OpenAI tools.
• 100 million weekly active users.

 

 

GTP-4 Turbo - A new GPT model

GTP-4 Turbo is a new model that improves on GTP-4 in some major ways.

It will be smarter than GPT-4, but it will also be cheaper. A LOT cheaper. In terms of API pricing, GPT-4 Turbo is 3x less expensive per input token, and 2x less expensive for output tokens.

In other words, using GTP-4 Turbo will most likely cost less than half (OpenAI's own estimates is 2.75 times cheaper) of what it would cost to implement GPT-4 into a product. And that's with better performance than GPT-4.

 

It will cost 1 cent for 1,000 prompt tokens, and 3 cents for 1,000 output tokens.

 

Right now, GTP-4 hasn't prioritized speed because they wanted to keep the price down. But speed is the next thing they will work on. The speed should improve "soon".

 

 

 

 

 

 

 

The six major improvements for developers

 

1 - Context length.

GPT-4 supports up to 8K (and in some cases up to 32K) of "context length" (number of tokens). This new model, ChatGPT-4 Turbo, supports up to 128K of tokens for context length. For comparison, this new model could keep a 300-page long book in memory for context. This also means it will be less inclined to lose accuracy as your conversations grow very long.

 

 

2 - More control.

They have implemented more features in regard to what their models outputs. For example, you can now flip a toggle that makes sure that the model will output a valid JSON format.

It can now also support calling several function calls inside the same single message. Before, if you asked it to "raise my windows and turn the radio on", it might only have raised the window. Now it will properly do both things at once.

"Reproducible outputs" is another new feature, and it is launching today. What it does is allow you to input a "seed parameter", and the model will try and format its future outputs after your seed.

They are also adding logprobs support.

 

 

3 - Better knowledge

The ChatGPT platform will be able to retrieve new information. You will be able to feed the model with your own information, such as a database, and add it to your program you're building using their APIs.

They have also updated the models' knowledge. It used to only contain information from before September 2021, but now it is updated with knowledge up until April 2023. They have also said that they will try and never let ChatGPT get so outdated ever again. They want it to have as much new information as possible, so they will keep feeding it new information.

 

 

4 - New modalities

They are adding API support for Dall-E 3, GPT-4 Turbo with Vision, and their text-to-speech model starting today.

 

The new text-to-speech model seems really powerful. You could probably still pick up on some stuff and figure out that it's an AI model generating the voice, but it seems to be good enough to not matter for legitimate use like language learning.

They are also releasing an update to their open-source speech recognition model "Whisper". The new model is called Whisper V3 and features updated support for several languages. Which ones remains to be seen.

 

 

5 - Customization 

Companies will now be able to work together with OpenAI to create a custom model. OpenAI will help companies with the development and training of the custom model.

OpenAI pointed out that they won't be able to do this with many companies, and it will be expensive, but they are inviting companies who really want to make a big push with AI to reach out to them.

ChatGPT 3.5 now also supports fine-tuning in the 16K model.

GTP-4 now also supports fine-tuning (in an invite-only experimental API). Not sure if GPT-4 Turbo will support fine-tuning but it doesn't seem like it.

 

6 - Higher rate limits

All GTP-4 customers will get a 2x increase in their API tokens per minute rate limit.

Customers will also be able to further increase their rate limit and get a direct quota for how much it will cost in their API account settings page.

 

 

 

 

Copyright Shield

OpenAI will now step in and defend their customers (ChatGPT Enterprise and API customers) in eventual legal battles over copyright claims. They will pay legal fees for you.

They also wanted to take that opportunity to remind everyone that they do not use data submitted from Enterprise customers or API customers to train their models on.

 

 

 

Cheaper GTP-3.5 Turbo 16K

OpenAI is lowering its pricing of GPT-3.5 Turbo 16K. Inputs are 3x cheaper and outputs are 2x cheaper than before.

From $0.003 per 1000 input tokens to $0.001 per 1000 input tokens.

From $0.004 per 1000 output tokens to $0.002 per 1000 output tokens.

 

This means that the 16K model of GTP-3.5 Turbo now costs less than the 4K model. This also applies to the fine-tuned models.

 

 

ChatGPT improvements

Despite this being a developer conference, OpenAI said that they will be making some improvements to ChatGPT as well.

 

First of all, they will update the GPT-4 model to GPT-4 Turbo for ChatGPT Plus subscribers.

 

Secondly, you will no longer need to select "Browse with Bing", "Dall-E 3" or whatnot from the dropdown menu. It "will just work" automatically from now on. If you ask it to draw an image, it will know you want to use Dall-E 3 without you having to select it.

 

 

GPTs - Customized versions of ChatGPT

GTPs are what OpenAI calls their new customized versions of ChatGPT. The idea is that someone (with seemingly quite little knowledge) will be able to make a custom ChatGPT version, that will behave in a specific way. You will be able to build your own GPT, using ChatGPT and natural language.

 

One example they showed was Code.org who has designed "lesson planner GPT".

It's a customized version of ChatGPT which is aimed at teachers trying to teach middle-school children coding. From what I understand, this ChatGPT version knows about Code.org's curriculum and intended target audience and will tailor its outputs based on that. So if you ask it what a for loop is, it will know that you're talking about programming, and give examples that will be appealing to middle schoolers, such as explain things using video game characters.

These GPTs can be integrated (and are in essence extensions) with their plugins. You can tell a GPT to pull information from documents you feed it. In the demo OpenAI uploaded a video lecture to a GPT and told it to pull information from the video, which it did.

 

 

 

These GTPs will be able to become private GPTs, create sharable links to your GPTs, or if you got an enterprise account you can limit a GPT to only being accessible from your own company.

There will also be a marketplace for GPTs. The most popular and powerful GPTs will get a portion of OpenAI's revenue. I wasn't able to decipher if this is similar to the app store where users pay let's say 10 dollars for an app, and some portion goes to the store owner, or if this will just be OpenAI giving away their own money to developers who publish stuff for free, just to incentivize people to create GPTs and create an ecosystem around that.

 

 

 

 

 

My thoughts

I will add things as they get announced. It will be interesting to see what happens though. 

 

Sources

Watch the opening keynotes:

 

 

 

 

 

 

 

[colonel_mortis]

I did not see them cutting the price of GPT-4 (even if it is technically a new model). GPT-4 was fairly expensive compared to GPT-3, but compared to the human-intensive tasks that it was able to help automate, it was nothing, and OpenAI are by far the market leaders at the moment (arguably in terms of tech, but definitively in terms of adoption). It'll be interesting to see what new use cases people end up building on top of this.

 

Having 3.5-turbo-16k be cheaper than 3.5-turbo-4k is also an interesting move, I wonder if they're just trying to get existing customers to keep paying more but attract new customers with lower prices (or maybe they just didn't think about it, or didn't announce it).

[wanderingfool2]

What bothers me is that OpenAI should have really been forced to change their name when they essentially went against the whole concept that had led to their name.

 

They are no longer open, they keep things close sourced (where at least some of the original intent was to keep it open) and operate it as a for profit company now essentially (sure capped, but when they have had billions of investment their capped means something like $100 billion) and ontop of that they already are censoring input based on certain stances.

 

I think things could be so different if they opened up the source, instead of controlling the API's.

[Kisai]

14 hours ago, wanderingfool2 said:

What bothers me is that OpenAI should have really been forced to change their name when they essentially went against the whole concept that had led to their name.

 

They are no longer open, they keep things close sourced (where at least some of the original intent was to keep it open) and operate it as a for profit company now essentially (sure capped, but when they have had billions of investment their capped means something like $100 billion) and ontop of that they already are censoring input based on certain stances.

 

I think things could be so different if they opened up the source, instead of controlling the API's.

A name is just a name. OpenAL, OpenCL, OpenGL, OpenSSL are just trademarks. Just because "Open" is in the name, doesn't mean it's open source, free, or can be held accountable.

 

That said, viable LLM's are not small enough to run on a GPU you'd have in a typical computer, We probably won't get there before desktops have 512GB of VRAM, or some new development in CPU inference that allows the LLM to be run entirely on the CPU. Still, your typical desktop doesn't have the capability to have 128GB, let alone 256, so we're not reaching that any time soon.

 

ASR, TTS and VC can all be done on a 16 or 24GB GPU, though training an ASR or TTS still requires a level of GPU compute power that isn't available. If you don't need realtime ASR or realtime TTS, you can run them on current CPU's. They will just take seconds, or even 10's of seconds, where as the GPU will have done it in about 50ms.

 

Dall-E/Stable diffusion stuff requires an unprecedented amount of compute power to train, but inference can be done on an 8GB GPU. Doing it on a CPU is probably not going to be fun.

 

A lot of the problems with "what hardware can run that" is really a question of "how big can the model be that still requires a cloud compute, thus pay money to use".  LLM's need a lot of memory because they really large models. Nothing else does because the output is a lot closer to a JPEG. Like your typical ASR operates on 16khz audio, TTS on 22khz, generative images 512x512 pixels, and then need some kind of upsampling algorithm to make it "less obvious it's an AI"

 

The emphasis is "less obvious" because no upscaling algorithm works well enough to work in all cases. Like DLSS in games might pass if the general visual quality is already kinda adaptable to it, but run that on videos (eg cartoons, or television) and it's a mixed bag of "oil painting" or "jpeg compression set to 60"