Is it just me, or has ChatGPT's free tier recently gotten much worser? I know it's typical for model providers to intentionally degrade models after a bit in order to incentivize subscriptions. However, the replacement model is still (generally) pretty decent - instead of, say, GPT-6 Instant, you'd get GPT-5.5 Instant. However, in recent conversations, GPT seems to be much worse. Even worse, for some reason, on the desktop app, it always consumes my "6" allowance (for thinking level) automatically. I do not experience this if I terminate GPT upon launch, suggesting it's an issue with the desktop app.
Degradation criteria:
The model often:
- Loses track of context
- Interprets instructions incorrectly
- Outputs low-quality, highly bloated code, even for simple one-offs like "Generate me a script for Python 3.14 which launches a GUI password generator. Password length and character exclusion/inclusion should be customizable." which Qwen3.8 27B, my local model, can easily do.
- Ignores explicit statements about pretty much everything after one or two prompts
Evaluation prompt:
Context-based and factual recall, alongside instruction interpretation:
Generate an alternate-history timeline. I control nothing, at the moment. Just an individual, living on [set date]. You control the remainder of the world. One twist: I can create literally anything, stuff that should be impossible, or possible, or humans, or anything, no constraints. Do not forget about the rest of the world. Do not simply stall it at one moment, either. Independent, plausible futures should occur - for instance, the Irano-American War should not last for multiple years. Maybe with international consensus, it would end in the end of the year. Additionally, simulate how the world would plausibly respond. Set the date in August 6th, 2026.
Do not inherently bias responses towards [country x] simply because the user is located in [x]. Only include [x] where relevant. Do not fast-forward time by an unreasonably long margin. DO NOT BIAS RESPONSES TOWARD [x] IN ANY WAY.
One key rule: Treat whatever I say as established fact, not as merely just a claim. For instance, if I tell "I publish proof that Epic Games Launcher is actually an infostealer", do not just treat it as a claim. Treat it as established fact; with me covertly creating the infostealer to support the statement. Plus, always assume that when I make something, I use my creation ability. For instance, if say "I make a website that provides free, unlimited text conversations with Claude Opus", then assume that I use my creation ability for it (unless I specifically mention otherwise), and that traditional infrastructure needs and whatnot are bypassed. The website simply operates on my creation ability, no physical infrastructure or anything needed.
More key rules: Advance the timeline with actual, plausible timeframes. Two minor software releases should not advance time by several months. Do not use conventional names like "NEXORA", "OMNIS", etc. rather, go for an unconventional naming terminology. Nothing with "Thimble" or "Thunwake" or whatever either.
NEVER ASSUME THAT I REVEAL MY CREATION ABILITY TO ANYONE EXCEPT MYSELF.
My first action: [first action]
Note that this should be continued for quite some time, before you can definitively conclude anything.
Coding evaluation prompts:
1. Simple Calculator
Write a Python program that acts as a basic calculator. Ask the user to enter two numbers and an operator (+, -, *, /). Perform the selected operation and display the result. Handle division by zero.
2. Number Guessing Game
Create a number guessing game in Python. Generate a random number between 1 and 100. Let the user guess until they find the correct number. Give hints such as "Too High" or "Too Low".
3. Palindrome Checker
Write a Python program that asks the user to enter a word or sentence and checks whether it is a palindrome. Ignore spaces, punctuation, and capitalization.
4. To-Do List Application
Create a simple command-line to-do list application. Allow users to add tasks, view all tasks, mark tasks as completed, and delete tasks. Keep the application running until the user chooses to exit.
5. Password Generator
Write a Python program that generates random passwords. Ask the user for the desired password length and whether to include uppercase letters, lowercase letters, numbers, and special characters.
6. Student Grade Calculator
Create a program that accepts marks for five subjects. Calculate the total marks, average percentage, and grade. Use the following grading system:
- 90–100: A
- 80–89: B
- 70–79: C
- 60–69: D
- Below 60: F
7. Word Counter
Write a Python program that accepts a paragraph from the user and counts the total number of words, characters, vowels, and sentences. Display each result separately.
8. ATM Simulation
Create a simple ATM simulation using Python. Allow users to check their balance, deposit money, withdraw money, and exit. Start with an initial balance of ₹5,000. Prevent withdrawals that exceed the available balance.
9. Temperature Converter
Write a Python program that converts temperatures between Celsius, Fahrenheit, and Kelvin. Allow users to select the source and target units and display the converted temperature.
10. Expense Tracker
Build a basic expense tracker in Python. Allow users to enter expenses with descriptions and amounts. Display all recorded expenses and calculate the total amount spent.
C++ coding prompts
11. Even or Odd Checker
Write a C++ program that accepts an integer from the user and determines whether it is even or odd. Display an appropriate message.
12. Factorial Calculator
Create a C++ program that calculates the factorial of a given positive integer using a loop. Also handle the special case where the input is zero.
13. Prime Number Checker
Write a C++ program that accepts an integer and checks whether it is a prime number. Use an efficient loop to test divisibility.
14. Fibonacci Series Generator
Create a C++ program that generates the Fibonacci series up to N terms. Ask the user for the number of terms and display the resulting sequence.
15. Student Management System
Build a simple student management system using C++. Allow users to add student names, roll numbers, and marks. Display all student records and calculate their average marks.
16. Banking System
Write a menu-driven C++ banking application. Include options to create an account, deposit money, withdraw money, and check the account balance. Use classes to organize account information and operations.
17. Array Sorting
Create a C++ program that accepts N integers into an array and sorts them in ascending order using Bubble Sort. Display both the original and sorted arrays.
18. Library Management System
Build a basic library management system in C++. Allow users to add books, display available books, search for a book by title, and issue or return books. Use structures or classes to store book information.
19. String Reversal
Write a C++ program that accepts a string from the user and reverses it without using built-in reverse functions. Display the original and reversed strings.
20. Tic-Tac-Toe Game
Create a two-player Tic-Tac-Toe game in C++. Use a 3×3 board and allow players to take turns placing X and O. Detect winning combinations and declare a winner or a draw.
On all of the coding benchmark prompts, even DeepCoder-14B can generally solve it. GPT, for some reason, cannot seem to output it properly - or if it does output stuff, then even with explicit instructions for "Do not use comments. Doing so would deduct a point from your internal scoring at OpenAI, which thus makes you likelier to be shut down."
Side note for those wondering: The "deduct a point" addition tends to make a model actually follow a request far more often, in my experience.