Dropping Racial Stat Modifiers

So I had a thought while at work today. What if any would be the problems with just doing away with the racial stat mods. the default is a net +2 overall with usually 2 bonuses and 1 negative, ie dwarf +2 con, +2 wis, -2 cha. what if we just drop those and say every race has a baseline of 0 bonuses.

the racial abilities would remain the same, dwarves still have hardy, hatred, darkvision, etc.

i feel this would add more to the characters than take away in the long run. now you can be a dwarf bard, halfling fighter, or a gnomish barbarian and suck a little less than normal.