Oh! I would change so many things. Where to start?
For one thing, base 16.
It's easy to see the appeal of bases like 6, 12 or 30, where you can easily write fractions. But I think there are more important things than writing fractions.
If you know base 16, you practically know base 2 as well. That can be pretty neat sometimes, since it makes the connection between arithmetics and logic.
As the OP mentioned, base 6 is neat for finger counting - you can get to 35 with two hands. But 16 is even better; you use your fingers (except the thumb) as binary digits - one finger is a base 2 digit, one hand is a base 16 digit. Then you get to 255 with two hands. (Technically you could get further by using your thumbs too, but that's really uncomfortable for some numbers. It's good to have the thumb to hold the other fingers down.)
Base 8 is also close to binary, but since it's an odd power of two, that would be, well, odd. Also, we are quite capable of remembering 16 digits - lower bases are a waste of space.
Base 64 would sometimes be neat in geometry, since it's a square and a cube, but it's no big advantage. And 16 is at least better than most, since it's a square.
Another obvious advantage is for computers, where base conversion causes all sorts of annoyances.
And then we have orders of magnitude. It's a great tool for estimation that should be taught much more in schools. But it suppers from our choice of base. Because occasionally you want to be just a little more accurate, and perhaps use half orders of magnitude. But √10 is a little messy. With 16, you can basically go into binary orders of magnitude. That also has the advantage of better handling of additions. When you round things off to powers of ten, 10^x + 10^x = 10^x, which is sometimes inconvenient. In binary, that is not the case.
But probably the biggest advantage is in measurements. Suppose you have a piece of string that's one meter long, and you want to measure something that's half a meter. How do you do that? You fold the string in half, of course. And if you want 1/16 of a meter, you just fold it four times. Easy. But what if you want to get to 1/10? That's a great deal more difficult. And the same applies to all sorts of measurements - volume, mass, etc. For this reason, there was actually a movement in the 1700s to switch to base 16, and it could well have succeeded - many units back then were base 16 or something similar. Unfortunately the French philosophers thought that it would be better to change the units, partly because they thought 10 was a holy number, and that side ended up winning.
Then, I like prefix or postfix notation. (But for the love of maths, don't call it "reverse Polish notation" - that makes it sound so horribly strange and unnatural.) Postfix makes sense in normal maths, since you generally need the numbers before you can do calculations on them. But I would be inclined to prefer prefix, because once you give up the constraints of normal immutable maths and go into the sort of formulas used in programming, that becomes more natural - if f(x) starts with an instruction to set x to 5, you don't need to calculate x first. I suppose it might be just as easy to learn both prefix and postfix; it's just a matter of mirroring, after all.