It depends on your use and other considerations.....
for instance, are you using the rifle as a truck gun, do you use the rifle for hiking around, or is it a table gun or used mainly for targets or sniping at occasional varmints?
Truck guns need short barrels, and the king of the hill is the Thompson Contender with a 22" barrel. With standard rifles, the 22" is probably about right for this application. It seems to me that a 20" barrel will just about blow your ears out if you happen to shoot it without ear protection even in 223.
On a sporter that you are going to carry around, there is practically no weight difference between a 20" & 24" barrel. You may see a difference in how a rifle may shoot if you go below a #4 contour and a #5 is preferable, especially on Magnums. I would try and save weight on the stock and have a little bit heavier barrel for accuracy. I am not a fan of Ultra light rifles because I have owned so many of them, I am an accuracy geeky freek. Another consideration on 20" barrels is that some powders have considerably more muzzle blast than others along with sparks flying out the end of the barrel under low light conditions that may distract you from trying to kill a running animal.
Heavier Varmint rifles are a real joy to work with because of their weight and barrel length. When re-barreling a rifle, I like the Max heavy varmint contour and in a caliber like 223, I will make it 28"-30" long. There are several reasons for this, but I am not just thinking of the use of the barrel on it's first chamber, but on several re-chambers later on. Usually if you start off with a barrel that has very minimal freebore, as the leade starts getting long, you be able to have your gunsmith cut off 1 1/2"-2" and you will get perhaps double the barrel life that you intended on that barrel. The caliber that you start off with is of course has a bearing on how much you have to cut off when you rechamber. The Max Heavy Varmint countour has 6" of straight barrel, and you will be able to cut the barrel off at least three times on a 223.
Cleaning rod wear will also have a bearing on how many times you can cut off a barrel also, I learned this from cleaning every 50 rounds even with a rod guide that has a bushing on it.
When considering barrel length, you also have to remember that we get three speeds of barrels when we get a custom barrel, slow speed barrel, average speed barrel, and really fast speed barrel. When you discover that you have a barrel that is slow velocity, it's like discovering that your gril friend has a Tall Black Marine that she is seeing on the side...you want to throw that bitch away! It may be prudent to order the barrel longer than you like, find out what type of speed you are getting with your load, then cut the barrel back with an informed opinion of what kind of velocity that you will end up with. You can always cut the barrel off, you can never add barrel length back on.
As you consider contours for a heavy barrel Varmint rifle, the heavier longer contour does two things, it decreases muzzle climb allowing you to see your targets better and it moves the concussion of the blast further away from your head, both of which make the rifle more enjoyable to shoot.
Good luck!