What makes Bitcoin different from MasterCard, PayPal, and other payment networks that existed in 2008 (when Bitcoin was invented) is that Bitcoin was the world's first payment network that's completely decentralized. The MasterCard network is operated by MasterCard Inc., but there's no Bitcoin Inc. in charge of the Bitcoin network. Rather, it's a peer-to-peer network that maintains a shared transaction ledger called the blockchain.
Inventing a new currency was an unavoidable consequence of making the Bitcoin network fully decentralized. On a conventional payment network, the network owner promises to redeem balances for conventional currencies like dollars or euros. But there's no Bitcoin company, so there's no one to guarantee that Bitcoin balances will have any particular value. Instead, bitcoins float against conventional currencies, with their value determined by supply and demand.
And no, those physical "bitcoins" you see in a lot of pictures aren't what a bitcoin "really" looks like. Bitcoins are just entries in the Bitcoin blockchain. If you own some bitcoins, that means you have some cryptographic private keys stored on your computer, on an external drive, or printed out on a piece of paper somewhere. These keys allow people to spend bitcoin balances in much the same way that the password to your bank's website allows you to spend the balance in your bank account. But you can't withdraw bitcoins from the network the way you withdraw physical currency from your bank.