Tech News

Are people really going to pay $200 a month for OpenAI’s new chatbot?

On Thursday, OpenAI released a $200-a-month chatbot — and the AI ​​community didn’t know what to make of it.

The company’s new ChatGPT Pro program offers access to “o1 mode,” which OpenAI says “uses more computing power to get the best answers to the most difficult questions.” An improved version of OpenAI’s o1 reasoning model, the o1 pro mode should answer questions related to science, math, and coding “reliably” and “completely,” OpenAI said.

Almost immediately, people started asking it to draw unicorns:

And he designed a “crab-based” computer:

And wax poetic about the meaning of life:

But a lot of people on X seemed unsure that the o1 mode responses were, well, $200 level.

“Has OpenAI shared any concrete examples of commands that fail on normal o1 but succeed on o1-pro?” asked British computer scientist Simon Willison. “I want to see one concrete example that shows its benefit.”

It’s a reasonable question; after all, this is the most expensive chatbot registration in the world. The service comes with other benefits, such as the removal of rate limits and unlimited access to other OpenAI models. But $2,400 a year isn’t chump change, and the value proposition for the o1 mode in particular remains disappointing.

It didn’t take long to find cases of failure. O1 pro mode struggles with Sudoku, and stumbles upon an optical illusion joke that is obvious to anyone.

OpenAI’s internal benchmarks show that o1 pro mode performs slightly better than o1 standard on coding and math problems:

Photo credits:OpenAI

OpenAI ran a “robust” test on the same benchmarks to demonstrate the consistency of the o1 mode: the model was considered to have solved the question if it got the answer four times out of four. But even in these tests, the improvements were not dramatic:

OpenAI mode o1
Photo credits:OpenAI

OpenAI CEO Sam Altman, who previously wrote that OpenAI was in way “espionage is too cheap to measure,” he was forced to clarify several times on Thursday that ChatGPT Pro is not for the masses.

“Most users will be very happy with o1 in [ChatGPT] An extra section!” said to X. “Almost everyone will be better served by our free tier or Plus tier.”

So whose is it? Are there really people willing to pay $200 a month to ask game questions like “Write a 3-paragraph essay on strawberries without using the letter ‘e’” or “solve this Math Olympiad problem”? Will they happily part with their hard earned money without much assurance that the average o1 can’t satisfactorily answer the same questions?

I asked Ameet Talwalkar, associate professor of machine learning Carnegie Mellon and business partner at Amplifaya Partners, his vision. “It seems like a huge risk to me to increase the price tenfold,” he told TechCrunch via email. “I think we’ll have a much better feeling in just a few weeks about the anticipation of this performance.”

UCLA computer scientist Guy Van den Broeck was blunt in his assessment. “I don’t know if the price makes sense,” he told TechCrunch, “and if the thinking models are expensive it will be the norm.”

The good take is that it’s a marketing mistake. Describing the o1 mode as the best at solving “toughest problems” doesn’t mean much to prospective customers. And there are no vague statements about whether the model can “think long term” and show “intelligence.” As Willison points out, without specific examples of this supposedly improved capability, it’s hard to justify paying more, let alone tenfold the price.

As far as I know, experts in specialized fields are the target audience. OpenAI says it plans to offer a handful of medical researchers at “leading institutions” free access to ChatGPT Pro, which will include o1 mode. Errors are very important in healthcare, and, as Bob McGrew, former chief research officer of OpenAI, pointed out in X, better reliability is perhaps the main enabler of o1 pro mode.

McGrew also considered the o1 pro mode to be an example of what he called “intelligence overhang”: users (and perhaps model creators) did not know how to get value from any “extra intelligence” due to the basic limitations of a simple, text-based interface. . As with other OpenAI models, the only way to communicate with the o1 pro mode is through ChatGPT, and – to McGrew’s point – ChatGPT is not perfect.

It is also true, however, that $200 sets expectations high. And judging by the initial reception on social media, ChatGPT Pro is no slam dunk.




Source link

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button