What’s this GPG/PGP business all about? According to Wikipedia:

Pretty Good Privacy (PGP) is a data encryption and decryption computer program that provides cryptographic privacy and authentication for data communication. PGP is often used for signing, encrypting and decrypting texts, e-mails, files, directories and whole disk partitions to increase the security of e-mail communications. It was created by Phil Zimmermann in 1991.

Don’t let its conception date fool you: PGP is still considered military-grade encryption and is extremely difficult to crack. Users generate a pair of keys, one secret and one public. Public keys are freely exchanged, which allows people to encrypt information that only very specific people can decrypt, using their secret keys. Someone must have your secret key and your passphrase you chose during key generation in order to decrypt your information.

For Mac users, GPGTools is the best and easiest way to get started (currently, GPGMail does not work on Mountain Lion yet, so no integration into Mail). For Windows users, Gpg4win seems to be the way to go.