People talk about “Unbalanced”, “OP” or “Overpowered” characters or monsters all the time, but there’s really no one standard for what that means. It can mean a few different things, in my experience.
The character in question is dominating the game, hogging the spotlight so that the other players feel left out. This is usually applied to characters that are way better in combat than the other characters, frequently one-shot-killing enemies that the rest of the group needs to spend several rounds whittling down. But it can also apply to characters that excel in some other area, like the thief that can sneak in, undo all the traps and retrieve the treasure while the other PCs just sit and wait at the tavern for the thief to return. Or the wizard who has a ring of teleportation, teleports without error into the vault, grabs the MacGuffin of the week and teleports back to the tavern. This kind of “unbalanced” character sucks because it makes the other people at the table feel useless. In my experience, the best way to deal with those characters is to make sure that there are challenges that the other PCs are better suited for: The wizard can teleport himself in, but it turns out the MacGuffin is actually split in two parts, and the second part is being held and carried by a powerful good guy who must be charmed by the bard or arm-wrestled by the fighter in order to give up their half of the MacGuffin.
The character or party is breezing through my challenges. We all know the problem: You spent hours and hours crafting an encounter with levels of tactical details with backup plans for failed attack routines, and hordes of minions and lietenants and BBEGs, and the enemy healers are staggered so they’ll be able to reach everyone on the field… And the players breeze through the combat in 2 rounds without getting hit at all. “They’re unbalanced! They’re too powerful for their level!” gets posted to Forums and Facebook groups, and the resounding advice that comes back is “Do a TPK! Meteor Strike them, steal their magical gear, throw them in a pit and pour acid down on them!”. This kind of unbalanced has its base in an unhealthy level of Adversarial GMing. People regard the GM as “against” the players, or the players as trying to “outsmart” the GM.
Regardless though of your opinions on the flood of PbtA games that are out there, they definitely get a few things right, and one of the big ones is that the GM should be a fan of the PCs. It’s not disruptive to your game when the PCs suceed at their goals. That’s the whole point of the game is for them to succeed at things. The problem cmes from when the PCs succeed without it costing them anything, and there’s no dramatic tension over whether they’ll succeed or not. The Antagonistic GM answer is for their success to cost them an all night combat where they get heavily injured, and possibly take a loss of a team member, and that’s certainly one form of sacrifice that can be interesting. But if you follow the rules of the game, and they don’t make that kind of sacrifice, your job as GM is to figure out what other sort of sacrifice can be interesting. Maybe while looting the bodies, they discover that the BBEG has already killed the King and thrown the kingdom into disarray, and the plot now (instead of defeating the BBEG’s army) is to raise the king from the dead and re-unify his country behind him. Or maybe the whole BBEG fight was just a distraction, and the dungeon has sealed behind them, and only his arcane signature/voiceprint/soulstone will unlock the doors. The point is to say “ok, you solved the combat problem, but now you’re faced with a different problem that can’t be solved by killing things.”
This comes up a lot from “Grognards” when they look at the PHB and see things like a Dragonborn PC who has a breath weapon at 1st level. “That’s too powerful for a PC!” they cry. Or a 1st level wizard with a 1d8 damage combat cantrip. “Unbalanced!” comes the cry. “I only allow Humans, elves and dwarves in my game! And any damage-causing spell is at least 1st level! If they don’t like it there’s the door!” But realistically, neither of those is unbalanced in any meaningful way. The fighter can do 1d8 damage (+ a strength mod!) with his sword every round until he runs out of hit points. Why is it “unbalanced” for the wizard or the Dragonborn to do the same? What this cry of “Unbalanced” really means is that the GM has a particular vision for what the world “looks” like, and is frustrated that the rules as written don’t provide the same vision, and especially that the PCs aren’t sticking to that same vision. This is just a problem of mismatched expectations. Most of these problems can be solved by a “session zero” where the GM outlines their vision for where the game should go, and the players can describe what they want out of their characters, and everyone can come to an agreement about what the vague outlines of the game should look like. This isnt just some happy, unicorns and rainbows love fest, everyone at the table should be challenging everyone else to improve things: “Oh, that PC sounds cool, but wouldn’t it be better if she was actually a member of a hidden dynasty who believes they’re the rightful rulers of the land?” or “I don’t really want to have my PC start as a dirt farmer, can we say he’s a veteran of a war who has been convalescing after an injury, and is slowly getting back up to fighting speed?”
Session zero is a great time to practice your “Yes and"s and “No, but"s.