[xmlsec] mscrypto: loading tests/keys/rsakey-win.p12 no longer works

[Miklos Vajna]

Hi,

I'm experimenting with running the equivalent of 'make check' for an
MSVC build with the mscrypto backend, as part of this I ran into this
issue. E.g.

make check-crypto-mscrypto XMLSEC_TEST_NAME="enveloping-sha256-rsa-sha256-relationship"

fails for the signing part with this:

func=xmlSecMSCryptoKeyDataAdoptCert:file=..\src\mscrypto\certkeys.c:line=449:obj=unknown:subj=CryptAcquireCertificatePrivateKey:error=4:crypto library function failed:MSCrypto error: -2146885621: 0x8009200b: Cannot find the certificate and private key for decryption.
func=xmlSecMSCryptoCertAdopt:file=..\src\mscrypto\certkeys.c:line=912:obj=unknown:subj=xmlSecMSCryptoPCCDataAdoptPCC:error=1:xmlsec library function failed:
func=xmlSecMSCryptoAppPkcs12LoadMemory:file=..\src\mscrypto\app.c:line=545:obj=x509:subj=xmlSecMSCryptoCertAdopt:error=1:xmlsec library function failed:
func=xmlSecMSCryptoAppPkcs12Load:file=..\src\mscrypto\app.c:line=451:obj=unknown:subj=xmlSecMSCryptoAppPkcs12LoadMemory:error=1:xmlsec library function failed:
func=xmlSecMSCryptoAppKeyLoad:file=..\src\mscrypto\app.c:line=146:obj=unknown:subj=xmlSecMSCryptoAppPkcs12Load:error=1:xmlsec library function failed:

I've bisected the .p12 loading failure and that lead me to
c098aed450a8ae272efad1fdeaa2450b67a2f46f. So it seems the problem is
that either the private key is persisted, or by the time
CryptAcquireCertificatePrivateKey() would read the private key, it would
read it from the .p12 file again, but at that point it no longer
remembers the provided password.

Now my question: my understanding is that .p12 files are always
password-protected, so this sounds like currently loading .p12 files
with the mscrypto backend in the app just doesn't work. If so, I wonder
if there is a way to have some 3rd version that allows persistence, but
only till the app runs, not after it (so there would be no leftover
files, nor the failure with CryptAcquireCertificatePrivateKey().

I've tried two things:

- CryptAcquireContext() has a CRYPT_DELETEKEYSET flag which may be
  useful here, but I don't know how to get the container/provider of the
  HCERTSTORE to use it for the private key loaded from the .p12 file (if
  this goal has any meaning :-) )
- CertDeleteCertificateFromStore() sounded promising, I tried calling
  that at the end of while() loop in
  xmlSecMSCryptoAppPkcs12LoadMemory(), but that did not eliminate the
  problematic temp files in the mentioned folder in the
  https://bugzilla.gnome.org/show_bug.cgi?id=633924 bugreport.

Considering all the above: I wonder if there is a way to handle this
situation (loading private key/cert from .p12 files for test purposes
without creating temp files) properly, or should we just say that the
current situation is broken enough that just going back to not using
PKCS12_NO_PERSIST_KEY is already an improvement?

Thanks,

Miklos