A sidequest isn’t all that different from any other small-scale plot in terms of lifecycle. Where it mostly differs is in its purpose. In D&D a plot’s job is to form the skeleton of a story, letting the DM and players add musculature, organs, and whatever bits of a body contribute toward this analogy. A good sidequest serves the same role with a few tweaks that make it a sidequest beyond simply being optional. These tweaks put some restrictions on the plots we can use, but they also made a sidequest distinct enough from the main campaign to serve as an effective temporary break.
For this post I’m going to use the 4E DMG adventure breakdown because I think it’s the most direct, concise way of describing an adventure in the same way that 4E is a direct, reasonably concise game to play. Your sidequest may omit some of these parts for various reasons (and going over why and how could be another post).
The first part of any adventure is likely the hook, and a sidequest is no different. You have to give the party a reason to stop what they’re doing and do something else, or to risk themselves on an adventure during what would otherwise be their downtime. Normally a sidequest has a lesser hook because of its scope. While it’s hard to build an entire campaign around “the party gets more money”, creating a small adventure in search of loot is both normal and acceptable. You can also have an NPC ask the players to save a small town, or a single business or a single person, and you don’t need to explain how that translates into a threat against the nation, the plane, or the nature of existence.
A sidequest hook also lets you put the focus on a specific character. Normally a hook has to involve the entirely party so everybody is invested in the story. A sidequest, as an aside to the main plot, doesn’t. You can make the hook about one character’s goals, backstory, friends, or arc and have the rest of the party (willingly or begrudgingly) along for the ride. You could also flip it and make the sidequest about one of your characters, like an NPC the party has already met, as long as the character doesn’t overshadow the party. The point is that sidequest hooks tend to be easier than campaign hooks because you know you don’t have to use them to kickstart a long-running plot, just a side story that the party can explore at their option.
Players must make choices in a sidequest the same way they do in any adventure or session, and those choices must matter, but there’s a wrinkle. In a campaign proper you’re not supposed to present the players with a choice that has a right answer and a wrong answer. You set up a decision, give the players information, let them pick an option (or two, etc.), and let them live with the consequences, but those consequences rarely include “a PC dies” or “the world ends” or anything that would cause the main plot to be a failure.
Sidequests can present plots with wrong answers. If the players answer the riddle incorrectly and blow up the dungeon, they won’t get the artifact sword that prompted the quest in the first place, but the failure doesn’t stall the game. That said, it’s not good practice to do it frequently, and you should prepare for players being frustrated if they put in a lot of effort or spent a lot of resources. It’s better to give them some lesser reward than nothing at all.
The challenges in a sidequest remain the same. You still want to build interesting fights, avoid frustrating mechanics and storylines, use varying encounter types (combat, puzzle, diplomacy, etc.), and keep an eye on how the players react to them all. If you’re using the sidequest to explore a specific character you can build encounters at which they excel (or at which they fail, so they need the party’s help). Here you may want to err on the side of safety with your challenges. Killing a character is rough, but killing a character on a B-plot that doesn’t even affect the large-scale campaign is mortifying.
All plots need excitement, and I would argue a sidequest needs it more than an average session. A sidequest, as a short adventure, goes on a truncated arc. You can’t give the players two boring setup sessions leading to a knock-down drag-out fight because that encompasses almost all of the quest’s runtime. You need to hit the players with something fun enough that they’re happy they went on the sidequest, and if anything it’s easier to do that because you only have to manage the pacing for a session or two.
You could give the players a set piece that doesn’t fit in the normal campaign, like a combat with unusual rules or enemies or a minigame that stretches what a D&D campaign entails. For example, one thing I’ve always wanted to run is a Dynasty Warriors-esque zone-capturing encounter, where the players have to assault enemy keeps to stem the tide of reinforcements while shoring up their own resources. They have to decide whether they want to blaze ahead in a difficult battle, hang back and defend while the allied NPCs do most of the heavy lifting, run around and grab important locations to enhance their battlefield position but leave attack and defense to the NPCs, etc. It’s not easy to organically build a campaign toward a small-scale, but broad within that scale, battle between opposing factions, and the one time I tried to put this into a campaign the players deftly avoided it entirely. It’s something I’ll have a much easier time putting into a one-off session.
Usually the best way to peak this excitement is to do it in the climax, which should be meaningful in terms of the sidequest but not better than similar climaxes in the main plot. The players shouldn’t say the best part of your campaign was the time they fought the orge raiders by jumping back and forth across a stampeding elephant horde if it happened in the session you threw together because you needed to stall another week to design the big bad’s fortress (and if they do, there’s a learning moment about what sort of encounters your players really want). You want a more local finale, one that wraps up the sidequest but doesn’t overshadow the rest of the game.
It may seem like I’m contradicting myself here. I say to give the players an off-the-wall set piece or combat, but not one so off-the-wall they like it more than the main plot. Here’s where having a mutable medium like tabletop gaming comes in handy. If you do give the players an encounter they adore in the sidequest, don’t consider it a lost cause. Instead add a similar encounter to the campaign at a later point, perhaps ramping it up a little. Now that sidequest that threatened to usurp your pacing wasn’t the accidental high point of the campaign, it was foreshadowing. Reward the players for their surprising skill under unusual circumstances, then hit them with the escalation.
If you’re intentionally giving the players filler, you may be tempted to avoid having a meaningful victory. But even in filler, you want the players to know they didn’t just waste their time. Give them an emotional reward rather than a physical or monetary one. Even if they didn’t get any tangible reward they still saved the village, or kept the warlock from opening the devil portal, or prevented Team Rocket from taking over a corner of the Safari Zone. The players know they did good, they’re happy, and they can move on.
With a proper sidequest you could consider the experience and loot to be its own reward. I like also giving the players something interesting enough that they’ll keep it around. Getting a +2 greatsword from the dead orc warlord is neat but forgettable. If the mayor of the town threatened by the warlord grants the party her ancestral sword which has the same game statistics but also leaves a trail of smoke whenever a character swings it, that’s better, and the party will be more likely to work with the sword and less likely to sell it for pocket change later. The rewards for a sidequest can be perfectly in line with main plot encounters of the same difficulty, or even worse, but if they’re fun the players will enjoy them more.
For further reading, TV Tropes has a pretty extensive list of sidequest examples and a few other pages for specific sidequest types. This can give you a good foundation and a lot of ideas what how broad sidequests can be, but there’s no substitute for doing it yourself. As with anything in D&D (or training Amiibos), the best way to learn it to try it and get it wrong.