A lot of the times, when you hit turbulence at altitude in an airliner, it's because of this. If you watch the map display closely on your seatback you can actually see it happening: you'll feel a big bump or jolt and then your speed readout might be 50mph less and your altitude might be 2000 ft less.

As other people mention, it doesn't happen immediately - it's like merging onto a very bumpy freeway.