There are a number of factors that effect public key size choice for use in the DNS security extension. Unfortunately, these factors usually do not all point in the same direction. Choice of zone key size should generally be made by the zone administrator depending on their local conditions.
For most schemes, larger keys are more secure but slower. Given a small public exponent, verification (the most common operation) for the MD5/RSA algorithm will vary roughly with the square of the modulus length, signing will vary with the cube of the modulus length, and key generation (the least common operation) will vary with the fourth power of the modulus length. The current best algorithms for factoring a modulus and breaking RSA security vary roughly with the 1.6 power of the modulus itself. Thus going from a 640 bit modulus to a 1280 bit modulus only increases the verification time by a factor of 4 but increases the work factor of breaking the key by over 2^900. An upper bound of 2552 bits has been established for the MD5/RSA DNS security algorithm for interoperability purposes.
However, larger keys increase the size of the KEY and SIG RRs. This increases the chance of DNS UDP packet overflow and the possible necessity for using higher overhead TCP in responses.
The recommended minimum RSA algorithm modulus size, 640 bits, is believed by the authors to be secure at this time but high level zones in the DNS tree may wish to set a higher minimum, perhaps 1000 bits, for security reasons. (Since the United States National Security Agency generally permits export of encryption systems using an RSA modulus of up to 512 bits, use of that small a modulus, i.e. n, must be considered weak.)
For a key used only to secure data and not to secure other keys, 640 bits should be adequate at this time.