Let's start with the acronym: GPS stands for global positioning system. Originally a DARPA-funded joint project of the Air Force and Navy, this satellite network tells ya where stuff is, like bombers and cruise missiles in decades past, or you as of mid-2000 when the government made GPS of decent accuracy available for civilian electronics. (It was available before then, but wasn't good enough for reliable turn-by-turn app.) The soul of GPS is the constellation of at least 24 satellites way out in orbit. Signals from four separate birds are usually needed for a standard GPS receiver to peg your position.
The GPS goods most people are familiar with are ones you mount in your car (though like we said, GPS will fit just about anywhere now) with the biggest players being Garmin, TomTom and Magellan. They used to be a lot more expensive, but now you can get basic namebrand models for not much more than $200, and cheap knock-offs for even less.
At a basic level, these all operate the same way, with variations in feature sets and UI: Your GPS receiver picks up signals from orbiting satellites and plots your position accordingly on pre-loaded maps. (The maps themselves typically come from one of just two companies, Navteq and Tele Atlas.) More recently, live traffic info (or something close to it) to avoid the Monday jam courtesy of an overturned 18-wheeler of pig lard has been the goal, with the pricey (but awesome) Dash Express delivering the up to the minute goods via GPRS cellular connection.
While GPS has gotten better in your car and on your wrist, the real excitement is its movements into cellphones and other gadgets such as cameras for location-based services (and maybe ads) and tricks like geo-tagging. Sprint's Instinct phone, for instance, makes a big a deal out of having real GPS while the iPhone has less accurate triangulation via cellphone towers, since being accurate to within several blocks isn't nearly as helpful as knowing where you are within a couple of meters. Friend finders and kid locators are options on pretty much every carrier.
As GPS modules get smaller and less power-hungry, you can expect GPS to keep showing up in ever smaller and crazier gadgets, since it'll be cheap and easy to cram it in. Manufacturers on everything from laptops to shoes are getting in on GPS mania, so even if you never owned a GPS device, odds are, you soon will.