[Dnssec-deployment] RRSIG for arpa expired

Matt Larson mlarson at verisign.com
Tue Jun 15 17:15:51 EDT 2010


On Sat, 05 Jun 2010, Casey Deccio wrote:
> Apparently the RRSIG made by arpa's KSK has expired (about 15hrs ago)

Here's a brief explanation of what happened.

VeriSign currently signs the .arpa zone under an interim arrangement
in which VeriSign not only holds the zone's ZSK and signs the zone,
but also holds the zone's KSK and uses it to sign the zone's key
sets.

The validity period for the .arpa zone's DNSKEY RRset's RRSIG ended at
23:59:59 UTC on Friday, June 4, 2010.  VeriSign monitors all aspects
of the .arpa zone creation, signing and distribution, and the
impending expiration of this RRSIG was noted by our monitoring systems
and sounded alarms.  However, at that time, in preparation for the
upcoming acceptance of TLD DS records for the signed root zone,
VeriSign and ICANN were conducting "dry run" submissions of DS records
through the normal, production root zone change process.  The DS
dry-run test plan called for submission of deliberately malformed DS
records by ICANN to confirm that such records could be detected.

Because our operations team anticipated malformed DS records and knew
that these would produce false alarms in the monitoring systems, the
team instructed our 24x7 monitoring personnel to disregard these
warnings.  At the time of the incident, events related to DNSSEC
signing of the .arpa zone and root zone change processing were
comingled in the monitoring system.  As a result, the 24x7 personnel
inadvertently disregarded monitoring events relating to the impending
expiration of the RRSIG for .arpa zone's key set.

VeriSign engineers identified the problem by 1630 UTC on Saturday,
June 5, and a properly signed .arpa zone was published to the stealth
master servers by 1715 UTC.

Going forward, VeriSign has taken several steps to prevent a
recurrence of a similar incident.  We have already separated events
related to DNSSEC signing of the .arpa zone in our monitoring systems
from other monitoring events as well as increased the clarity of the
events as they are alerted.  We have also instituted additional "out
of band" monitoring of DNSSEC parameters of the .arpa zone.

VeriSign is confident that the issues that led to the incident have
been addressed.

Matt


More information about the Dnssec-deployment mailing list