A bit that keeps dropping out of an impact driver is annoying, but once you understand how the front of the tool actually holds a bit, the fix is usually obvious. So let me walk through how the mechanism works, then we'll go through the causes in the order they're most likely to be the problem.
How the collet actually grips a bit
The business end of an impact driver isn't a chuck with jaws like a drill. It's a quick-release collet sized for a 1/4-inch hex shank. Inside that collet sits a small spring-loaded steel ball. When you push a bit in and let the sleeve snap forward, that ball drops into the groove cut around the bit's shank and locks it in place. Pull the sleeve forward and the ball can move out of the way, releasing the bit.
That one detail explains almost every case of a bit falling out: the ball has nothing to grab, or it's worn out and can't hold. Keep that picture in mind and the rest is straightforward.
The most common cause: the wrong bits
Often the bit is the problem, not the tool. Bits built for impact drivers have a machined groove, sometimes called a power groove, around the shank near the end. That groove is exactly where the collet ball seats. Cheap or older bits with a plain, smooth hex shank give the ball nothing to lock into, so the bit slides right back out under vibration.
The fix is simple. Use impact-rated 1/4-inch hex bits with the visible groove ring. Look at the shank: if there's a notch or groove cut around it, the collet can hold it; if it's smooth, it can't reliably. This single swap solves the majority of "won't stay in" complaints.
Next: the bit isn't seated all the way
Even a correct bit falls out if it never clicked in. Push the bit in firmly until you feel and hear the collet snap back over the groove, then give it a gentle tug to confirm it's locked. On some tools you don't need to pull the sleeve at all to insert, just press the bit straight in. If you're holding the sleeve forward while you push, you're keeping the ball retracted, which is why it won't catch. Press the bit in first, then release.
Short bits that sit too shallow
Stubby one-inch insert bits are a frequent offender. They can sit so far back that the groove never lines up with the ball, or they bottom out before the collet engages. The cleanest fix is a magnetic bit holder. You lock the holder's full-length shank into the collet once, and then swap short bits in and out of the magnetic tip without fighting the collet at all. It also keeps the screw stuck to the bit, which is a nice bonus on overhead work.
Debris in the collet
If you work in drywall dust, sawdust, or dirt, packed debris can keep the ball from dropping fully into the groove. Remove the battery, then blow the collet out with compressed air. Cycle the sleeve a few times to clear it. Don't spray lubricants or solvents into the tool; check your manual.
When the collet itself is worn out
Here's how to tell it's the tool and not your technique. Put a known-good, impact-rated bit in. Seat it fully. If it still drops out, or the sleeve doesn't snap back crisply, the detent ball or its spring is worn, and the collet has lost its grip. This is common on a high-mileage driver.
The good news is you may not need a new tool. On some models the quick-release collet is a service part; check your model's parts diagram or contact the manufacturer. Remove the battery before servicing the collet. If the tool is still under warranty, let the manufacturer handle it instead.
Run through these in order, correct bits first, and you'll often have it holding again in a couple of minutes.