The main danger for visitors to the city is theft (from public transportation, hotel rooms, and cars). Try to keep your passport and valuables in the hotel safe.
Taxis should be hailed over the phone rather than caught on the streets - the fare will be cheaper and you won't get scammed.
Tourists are often tricked into exchanging money on the street: after the tourist agrees to the exchange, pseudo-police officers approach him, extorting money "for silence".
Remember: in Budapest, it is forbidden for police to check wallets; if you are asked to do so, offer to show the "policemen" your documents or defiantly agree to be "arrested".
Beware of invitations from strange girls to visit a restaurant or bar with them (they are likely to be in cahoots with the administration and will "bribe" you, as they say).