Until the days of the blown exhaust, the FIA didn't have any issues about exhaust exit. So the re-writing of the regulations was aimed at eliminating the blown exhaust. So they wrote the new rules, believing that this change would eliminate this form of technology. But Formula One engineers, being smart and creative, found ways to still use the exhaust energy to assist in downforce.

This is a picture of the exhaust for the 2011 RB7, it's right down on the floor, attached to it. So they figured, raise it up a substantial distance, and aim it up. That will get rid of it, right? History has proven it to be false, it wasn't entirely successful. Here is the specific regulation.
Quote:
5.8 Exhaust systems :
5.8.1 With the exception of incidental leakage through exhaust joints (either into or out of the system), no fluids, other than those which emerge from the engine exhaust ports, may be admitted into the engine exhaust system.
5.8.2 Engine exhaust systems may incorporate no more than two exits, both of which must be rearward facing tailpipes, through which all exhaust gases must pass.
2012 F1 Technical Regulations 24 / 77 9 March 2012
© 2012 Fédération Internationale de l’Automobile
5.8.3 The last 100mm of any tailpipe must in its entirety :
a) Form a thin-walled unobstructed right circular cylinder whose internal diameter is no greater than 75mm with its axis at +/-10° to the car centre line when viewed from above the car and between +10° and +30° (tail-up) to the reference plane when viewed from the side of the car. The entire circumference of the exit should lie on a single plane normal to the tailpipe axis and be located at the rearmost extremity of the last 100mm of the tailpipe.
b) Be located between 250mm and 600mm above the reference plane.
c) Be located between 200mm and 500mm from the car centre line.
d) Be positioned in order that the entire circumference of the exit of the tailpipe lies between two vertical planes normal to the car centre line and which lie 500mm and 1200mm forward of the rear wheel centre line.
This is what they expected and were hoping for

This is what they got.

We have to remember that from first test, there was an incredible flurry of work behind the scenes, as each team had a good look at the competition, and made modifications at an incredible rate, constantly refining and improving the exhaust to extract more and more performance. If you look at the pictures of any car at the first test, and the end of the season, they have all been radically changed, and evolved.
Why not just have the exhaust exit at the extreme rear of the car? I can think of one good reason, safety. The last part of the car is the rear crash structure, intended to deform and die to absorb forces. Putting a hot metal pipe in that area makes it a very real risk from impact, and well it could harm course workers because of the high temperatures.

The last thing anyone wants is an exhaust pipe penetrating another driver's helmet in an accident.
So what's going to happen in the future? The 2014 regulations mandate a change, where the exhaust(s) have to exit behind the rear axle, are up high, and contained within a narrow region very close to the centerline of the car. That should work, but you can't dismiss the intelligence and ingenuity of the designers.