Coinbase has a "hot wallet" where 1-2% of coins are made available. 98-99% of bitcoins at anytime are stored in a "cold wallet" (offline) at locations across the world. To be truly safe, you can create your own "paper wallet" offline where no one else has access. Think of it as a dollar bill or paper currency. The Winklevoss twins store their BTC on computers in secure locations that never connect with the internet. You could create a brain wallet where you remember the keys. A lot of people just download the bitcoin-qt and store them on their computer. Make sure you encrypt your account if you do this! You can do all this stuff for LTC and most other altcoins. Note: some of the altcoins have sketchy qt wallets and are all-around sketchy.