Just to clarify the reason to buy bigger brakes as caseys et al were saying - and I apologise in advance for getting quite technical here but you already understand anyway - you need an increased rate of deceleration to the ABS actuation limit, as acceleration is the square of velocity, (or should that be the other way round? Probably not atall!), so the same goes for deceleration, that is, twice the speed requires four times the energy dissipation, so in other words, there is much more momentum in the car for what is only a little more speed and it's momentum that needs to be reduced, since you can slow down to say, 40 but still be carrying much more momentum than the guy next to you who braked earlier (or stronger) and can accelerate out of a corner, for example, while you struggle to stay on the track at the same speed leading in.
Think of momentum as another smaller car inside yours, where everything is fine as long as you're both going the same speed but yours is the only one with brakes, (hence the invention of seatbelts), so even as you brake, there's still a separate force pushing you along that needs to be tamed.
The faster you accelerate, the quicker the car builds momentum, which is what might catch you out if you're not careful and bigger brakes kill momentum much quicker than small ones. I know you said you'd not use the car on a track but if you did, you could brake later and keep more momentum going through a corner under control, thus giving you the winning advantage.
Besides, I have some 356mm D2 8 pots for sale!