I'll add some technical details to Eddie's answer.
Bitcoin uses a network of computers. These computers aren't controlled by a company but by lots of regular people who downloaded the Bitcoin client and keep it running in their home (usually 24/7).
These computers all talk to each other and basically act like a bank; each Bitcoin wallet is like a bank account. The balance of each account is stored in the so called blockchain, which is basically a ledger containing all transactions ever made, in order. Each computer on the bitcoin network stores a copy of this ledger file, which is about 150GB in size by now.
Anybody can create a wallet/account for free. When you do that, you just get two long numbers, a private key and a public key. When you buy bitcoin, you give somebody who already has bitcoin a bunch of dollars, and they in turn transfer bitcoin from their wallet to yours.
This transfer works like this: you give them your wallet's public(!) key. They take their own wallet's private key, and the two keys and the amount supposed to be transferred are converted into a transaction block. This block is now sent into the network. Lots of computers on the bitcoin network now check the block (make sure it's a valid transaction), and if the transaction is confirmed, it gets added to the end of the blockchain.
The balance in your wallet has now risen by the transferred amount, minus a transaction fee. This transaction fee is distributed among the people who mine bitcoin.