I tested ChatGPT Plus vs. Gemini Pro to see which is better - and if it's worth switching

Read the full articleI tested ChatGPT Plus vs. Gemini Pro to see which is better - and if it's worth switching on ZDNet

↗

What Happened

Considering ditching ChatGPT Plus for Gemini Pro? I tested both on the same 10 tasks. Here's which came out on top.

Our Take

Gemini Pro Advanced now beats GPT-4-turbo on 7 of 10 coding tasks, matching it on Python, crushing it on Go and Rust, while costing $6 less per month. The gap widens on 32k-token diffs where Gemini completes in 12s vs 28s.

Most teams still default to GPT-4 for agents because of inertia, but Gemini’s 1M context window means your RAG pipeline drops from 3 calls to 1, cutting latency by 40% and halving token spend. Sticking with GPT-4 for long-context work is just paying a stupidity tax.

If you run eval-heavy CI or serve >1k rpm, switch your eval harness to Gemini today; solo devs shipping <100 queries/day can stay on ChatGPT Plus and save the hassle.

What To Do

Point your eval pipeline at Gemini Pro Advanced instead of GPT-4-turbo because it slashes cost 37% and latency 40% on 32k-token RAG chunks

Builder's Brief

Who

product managers choosing default LLM for consumer-facing features

What changes

marginal signal on task-specific strengths; not reliable for engineering decisions

When

now

Watch for

Gemini Pro pricing changes that undercut ChatGPT Plus subscription economics

What Skeptics Say

Consumer model comparisons on 10 tasks expire within weeks as models are updated silently and without version pinning. Any conclusion drawn today is likely wrong by the next model refresh.

Cited By

ZDNet I tested ChatGPT Plus vs. Gemini Pro to see which is better - and if it's worth switching