[xmlsec] loading crypto engines as plugins, build changes, etc.

[Aleksey Sanin]

It turns out that the patch for loading crypto engines as plugins
exceeds 14,000 lines. I do not want to include such a big change
into the same release with another big change (mscrypto).
Thus I have decided to check it into the tip and do another 1.1.x
release soon. Of course, this means that all the changes would
go in both mscrypto branch and tip. The patches are attached
to corresponding bug:

    http://bugzilla.gnome.org/show_bug.cgi?id=121579

Bellow is the list of changes. Read on if you are interested, otherwise
you may skip to item 8 (this is important!).

0) Loading crypto engines as plugins.

I made the initial implementations for loading xmlsec-crypto libraries
as plugins. Since I still think that it might be not a good idea in some
cases (and to provide backward compatibility), the traditional linking
with static or shared xmlsec-crypto libraries is also supported.

The implementation is based on the libtool ltdl libray. It is compiled
into xmlsec thus there is no need to install a new separate library.
To avoid name conflicts with other libraries that might also use ltdl
library, I have changed all the names inside the ltdl files from "lt_*" or
"ltdl_*" to "xmlsec_lt_*" and "xmlsec_ltdl_*".

The DL support in xmlsec should work on any platform supported
by ltdl:

# |dlopen| (Solaris, Linux and various BSD flavors)
# |shl_load| (HP-UX)
# |LoadLibrary| (Win16 and Win32)
# |load_add_on| (BeOS)
# GNU DLD (emulates dynamic linking for static libraries)
# libtool's dlpreopen

However, currently it works only for ./configure based platforms (i.e. it
does not work on Windows). Supporting it on Windows is my next step
(hope it would be a matter of setting correct defines).  Also I am not sure
that this would work for none gcc compilers (for example, Sun Workshop CC).
If you have one of those then I would appreciate if you can try the current
CVS trunk and report your results.

The dynamic loading for xmlsec-crypto libraries (plugins mode) is compiled
in by default. You can disable it from ./configure command line using
these options:
    --enable-crypto-dl      enable dynamic loading support for 
xmlsec-crypto
                                          libraries (yes)
    --apps_enable-crypto-dl  enable dynamic loading support for 
xmlsec-crypto
                                          libraries in xmlsec command 
line tool (yes)
However, even if dynamic loading support is enabled in xmlsec library, you
would need to define XMLSEC_CRYPTO_DYNAMIC_LOADING in your
application Makefiles in order to enable it (also you would need to add one
additional call to load the xmlsec-crypto library you want). This 
ensures that
any existing code that does not want to use the new feature would not need
to change at all. I would describe all the details in a tutorial very 
soon (defenetly
before anything will be released). If you use static/shared library 
linking and
do not want to change this then you need to change *nothing* in your code
or Makefiles.

If you have any problems compiling xmlsec with DL support on your platform
then please submit a bug report. Meantime, you might get away if you specify
       "--disable-crypto-dl"
flag for ./configure script.

1) Changes in xmlsec command line tool.

To support this new functionality a new "--crypto <name>" option is 
added to
xmlsec command line tool. The "<name>" is the name of crypto library 
("openssl",
"gnutls", "nss", "mscrypto", etc.) or "default" to load default crypto 
library. If the
"--crypto" option is missing we load default crypto library.

Please note that this new option is available only when both xmlsec 
library and
xmlsec command line utility are compiled with enabled dl support. Otherwise,
you'll get an error "unknown option".

2) Changes in tests suite - make check.

If both xmlsec library and xmlsec command line utility are compiled with 
enabled
dl support then "make check" from top folder would run test suite for 
*all* available
crypto libraries. This is very nice and I really like it :)

3) Changes in rpms and *.pc files (todo).

 From now on I am going to compile rpms with enabled dl support. This 
would allow
me to put all non-crypto dependent code in xmlsec and xmlsec-devel packages
and use xmlsec-<crypto> and xmlsec-<crypto>-devel packages for 
xmlsec-<crypto> stuff.
This means that current xmlsec and xmlsec-devel packages would be splitted
in xmlsec, xmlsec-devel, xmlsec-openssl and xmlsec-openssl-devel packages.
If you use rpms then you would need to install at least one 
xmlsec-<crypto> package
to use xmlsec library and command line tool. Sorry for inconvinience but 
I feel that
this is a right thing to do. As before the default crypto library in my 
RPMs is openssl.

Currently we have separate xmlsec1-<crypto>.pc files and we do "ln -s" the
xmlsec1-<default-crypto>.pc file to xmlsec1.pc file. In the future I am 
going to change
this and provide an xmlsec1.pc file to configure xmlsec in plugins mode 
(of course,
if this mode is enabled). Similar change would be done for 
xmlsec1-config script.

4) Changes in Windows builds.

The Windows builds were modified to build multiple xmlsec-<crypto> 
libraries in one build.
Before the "--crypto" param for configure.js had one crypto library name 
and now it contains
a comma separated list of libraries where the first library is default. 
For example, if you
call configure.js with "--crypto openssl,mscrypto,nss" then all three 
xmlsec-openssl, xmlsec-mscrypto
and xmlsec-nss libraries would be build and xmlsec-openssl would be 
default. Currently there is
no DL support on Windows and "default crypto library" means that this 
crypto library is used
for xmlsec command line tool linking. In the future this would change 
and match the *nix
behaiviour.

Pelase note that this change is backward compatible and if you already 
have setup to build
xmlsec on Windows then you need to change nothing (unless you want to :) ).

Probably in the future we should make mscrypto the default crypto engine 
on Windows (Igor?).
Also it would be nice to include all the supported xmlsec-<crypto> 
libraries in Windows
binaries (again, Igor? :) )

5) Changes for documentation (todo).

There would be a new tutorial and API reference pages. As soon as 
possible :)

6) Changes for mscrypto branch.

Wouter, some of my changes might cause merge conflicts when you'll try 
to update your
sources from CVS. I am sorry about that but this should not be very bad 
(I expect that it would
be only one src/mscrypto/crypto.c file). I guess you'll quickly figure 
out how to fix this but
just in case, look at similar code for openssl or nss. And feel free to 
ask me questions :)

7) Wrapper for crypto libraries.

As before there is a minimal set of common functions required for xmlsec 
command
line tool. However, the new code allows adding additional "wrapper" 
functions if needed.
As I said before, I don't think that this is a really good idea. But if 
you think that there is
some "common" functions that could and should be implemented in xmlsec then
do not hesitate to send a mail to the list and we will discuss it. Just 
keep in mind that
the API should work for all existing and all future crypto engines 
supported by xmlsec :)

8) Help needed!

I have tested these changes on Linux (RH9) and Windows. However, I have 
some concerns
about other platforms (the dynamic loading is very platform specific 
feature):

    Sun Solaris with gcc (64 bit?)
    Sun Solaris with Sun's CC (64 bit?)
    HP-Unix
    FreeBSD
    Mac
    Everything else

If you can find 10-15 minutes to checkout current CVS trunk, run 
"./autogen.sh && make && make check"
and report about both positive and negative results then this would help 
to fix problems for your
platform *before* I would do next release.

Thanks to everyone,
Aleksey