i don't think its a matter to complain about it, and i certainly don't think you will like it either, if it will get changed then all of human classes will get magic defense which means your output damage on humans will get decreased(assuming you are a wizard), and vice versa. elves will get physical defense which will mean they will be a bit more tanky against physical type.
but each race has its own unique passives, humans gives mana and physical defense, hybrids gives physical damage and health, and elves gives magic defense and increase int/agility
casters(of elves race) suppose to have a lower physical defense in the first place because they are not focused on the physical stats which are strength and stamina (these 2 grant physical defense) so granting all defense wont change much to the casters tbh and you wont see the difference in damage taking. casters of the human race will do have a higher magic defense but the downside of it as i mentioned before, will grant the whole human class more magic defense which will decrease your own damage on them so not sure if this is a worth trade off.
also wont it damage the uniqueness of the classes themselves?
and a little further explanation, it wont only affect casters, it affects the whole race(its a race skill passive, not class) which will result in higher defense for everyone except hybrid race and some classes such as templar(will boost their magic defense), saint(will boost their magic defense) or prophet(will boost their physical defense) will get much more benefits out of it than the casters will since they tend to focus more on survivbility than other stuff and believe me, they don't need more boosts to their defense since its already hard to poke them down (good geared ones)
in the end of the day, i don't see the actual reason for changing it from physical defense or magic defense to all defense because it creates more problems than solving.