I agree with you it is annoying but i have an idea how to solve this out(wont be effective that much tho...), so just instead the buffs will be added at the start of the buffs bar, it will be added after your last buff.
for example:
1.png is when you buff/trans before the change(look at the potential growth buff)
2.png is when you buff/trans after the change(look at the potential growth buff)
hope you understood what i have meant XD