Netscape CA handling: When is a CA Not a CA?

Introduction.

Note: newer versions of Netscape have changed their behaviour and the main problem appears to be fixed. If you are running 4.06 or later of 4.5PR1 or later then check out the latest versions section.

The Symptoms.

When you click on the "invalid signature" box it will give you a reason for the signature failure in a security window. There are several common reasons.

• Message has been altered.
• CA is not trusted.
• Issuer Certificate is invalid.

If you get the "not trusted" message then you can go through the list of signers (Security->Signers) find the CA and set it to be trusted if you so wish. If you then reopen the message the message shows the usual "signed" box.

If you get "Issuer Certificate is invalid" then the CA will not appear in the list and you can't trust it. If you try to add the CA in question yourself then it will say the CA is already in the database: but it is not listed.

At this point the database is stuck in this state. You can still use it but the CA in question will be left unstrusted and invisible. A natural question is how the message was sent it the first place if the issuer is invalid. This will be answered in subsequent sections but first a bit of backround.

CA flag: X509 V3 extensions.

A user will possess their own private key. What is to stop an end user issuing certificates and pretending to be a CA themselves?

The answer is that most certificates these days are V3 certificates which can include various certificate extensions. One of these is called basicConstraints. It includes a flag called cA to indicate whether the certificate belongs to a CA or not. Thus the end user certificate will not have cA set and the intermediate CA (and root CA) will have cA set.

If an end user tries to impersonate a CA anyone attempting to verify such a bogus certificate can see it is not signed by a CA and take appropriate action.

This is a considerable simplification of the overall picture but it will suffice for now. A fuller description is given in the PKIX standards documentation for example.

Auto inserting CA certificates.

• If mail is read signed by an unknown CA.
• If a PKCS#12 file is imported and it contains an unknown CA.
• As part of the certificate enrollment process.

Untrusted (the term Netscape uses) is not really the right term. If a CA is auto added, but its parent CA is trusted, then the untrusted CA is trusted because it inherits trust from its parent CA. If a CA is set to be trusted then it is trusted even if the parent CA is not.

Installing CA certificates: anything goes.

Note I said certificate and not CA certificate. With this method you can install end user certificates, certificates with invalid signatures just about anything that vaguely looks like a certificate can be added and trusted as a CA.

When you actually try to use an invalid CA installed by this method all seems fine because it is trusted. If you send S/MIME mail with this CA then the poor recipient gets it added using the validity criteria and it is deemed invalid.

A whole organisation could install certificates for their users by first adding the CA certificate and everything seem fine because they all had the CA added as trusted. Only when they start sending signed mail to people who haven't installed the CA certificate does the problem become apparent.

Diagnosing the problem.

1. Open messenger and create a new message, click the sign box and save it as a draft.
2. Open the message in the Drafts folder and check it appears as signed.
3. Exit netscape.
4. Locate your user directory and move the files key3.db and cert7.db somewhere safe.
5. Restart Netscape and open the draft signed message again.
6. It should say that the signature is invalid.
7. Click on the box and check the reason.
8. If the reason says that the issuer certificate is invalid then you have a problem: as does anyone you send signed mail to: stop at this point and fix the problem. If it merely says that the CA is not trusted then you are OK.
9. Select the new CA in Security->Signers and set it to trust email users.
10. Reopen the message and it should now say just have the regular "signed" box.
11. Exit communicator are restore your key3.db and cert7.db files.

Solutions to the problem.

If you are using SSLeay then the normal method of generating a CA (CA.sh -newca) generates an invalid V1 CA. I have written certificate patching program called ca-fix that can be used to fix such certificates. For more information see the ca-fix documentation.

Latest Versions of Netscape.

The reason for this is that the S/MIME behaviour has changed recently. The root CA is no longer included with signed messages. This means that if you have an uninstalled S/MIME root CA the recipient has to add it manually.

I reported this as a bug and apparently it is an intentional change. Personally I think this is a very bad idea and I've requested that the old behaviour should be possible via an option: if you agree then I suggest you complain to Netscape. At best this is a kludge to avoid the problem though there are potentially other commercial reasons for its inclusion.

The good news though is that the problem appears to be fixed. If you now receive mail signed by an invalid CA then checking the signature can still give "issuer certificate is invalid" but an invalid CA is not added to the database: so you can add it manually.

As should be apparent the above test for the problem no longer works because the root CA is not auto added when you read signed S/MIME mail (because it isn't there!). There is however an easier way to diagnose if Netscape doesn't like your CA.
 

1. Select the CA it in Security->Signers.
2. Select Edit, uncheck the three "Accept boxes" and click on OK.
3. Click on Verify.
4. If it says just "Certificate not trusted" then all is OK. If it also says "Not a valid Certificate Authority" then it regards it as invalid.
5. Click Edit and retrurn the "Accept boxes" to their original state.