[xmlsec] extended character set encryption/decryption

[Russell Beall]

Hi Daniel,

I found my old playground where I hacked on the libxml library to get around this.  I finally whittled it down to line 4183 of parser.c in version 2.7.7.  

$ diff parser_russ.c parser.c.dist
4199,4200c4183
<                 if ((ctxt->instate != XML_PARSER_CONTENT) && (ctxt->instate != XML_PARSER_START)) {
---
>                 if (ctxt->instate != XML_PARSER_CONTENT)

The following modification also seemed to make a difference, but now when I comment it out, it doesn't actually seem to change the behavior, but I remember it being important.  I think it was an alternate way to get around the problem, but then I found the above method worked without me actually having to reassign the parser state.

13009,13012d12963
<      else if (node->type == XML_DOCUMENT_NODE) { 
<        // just set instate 
<        ctxt->instate = XML_PARSER_CONTENT; 
<      } 

I'm not sure anymore why this is occurring for this situation, or why this fix works around it, but I think it has something to do with the fact that we have a bit of an unusual use of xml encryption wrapped inside a soap message, but the content of the message is an entire escaped xml document, rather than just an encrypted section of the xml.  So it ends up trying to parse starting at a document node instead of just content.

Regards,
Russ.

On Apr 2, 2013, at 10:52 AM, Russell Beall <beall at usc.edu>
 wrote:

> Same behavior with 2.8.  Couldn't get 2.9 to compile (on Solaris 10).
> 
> I'll see if I can track it down again.  I remember locating it like two years ago and hacking in a patch, but then I didn't want to use the patch/hack, so I reworked the usage of the API.  I thought I sent an email about it to the list then, but I can't find it in my sent mail.
> 
> I note that separating the API by running xmlSecEncCtxDecryptToBuffer, and then xmlParseMemory on the result of that does not have any parse error.  This actually solved my problem as well, because then I can use xmlDocDumpMemoryEnc to force the encoding to US-ASCII so I can get it out of the C++ code and over into the python code where it is parsed and used.
> 
> Regards,
> Russ.
> 
> On Apr 1, 2013, at 8:03 PM, Daniel Veillard <veillard at redhat.com>
> wrote:
> 
>> On Mon, Apr 01, 2013 at 10:03:44PM +0000, Russell Beall wrote:
>>> Ok.
>>> 
>>> Maybe the code I used to use will still work for this, but when I upgraded the libraries, I had to switch from
>>> xmlSecEncCtxDecrypt
>>> to
>>> xmlSecEncCtxDecryptToBuffer
>>> 
>>> This was because decrypting packets like below produces a document error, perhaps within libxml:
>>> Entity: line 4: parser error : internal error
>>>     <USCID>5843020612</USCID>
>>>            ^
>>> func=xmlSecReplaceNodeBufferAndReturn:file=xmltree.c:line=573:obj=unknown:subj=xmlParse
>>> InNodeContext:error=5:libxml2 library function failed:Failed to parse content
>>> func=xmlSecEncCtxDecrypt:file=xmlenc.c:line=648:obj=unknown:subj=xmlSecReplaceNodeBuffe
>>> r:error=1:xmlsec library function failed:node=EncryptedData
>>> 
>>> This happened regardless of extended character sets.
>>> 
>>> Previously I would decrypt to a document and then do an xmlDocDumpMemory to get the data.
>>> 
>>> Does this ring any kind of a bell?
>> 
>> Honnestly no, this is issued inside libxml2 parser
>> I would need some kind of stack trace to understand what is going on
>> plus you are still using a relatively old libxml2 version.
>> 
>>> Maybe I need to run another upgrade of libxml or libxmlsec?
>> 
>> worth trying, 2.8.x or 2.9.0
>> 
>> Daniel
>> 
>> -- 
>> Daniel Veillard      | Open Source and Standards, Red Hat
>> veillard at redhat.com  | libxml Gnome XML XSLT toolkit  http://xmlsoft.org/
>> http://veillard.com/ | virtualization library  http://libvirt.org/
>> 
> 
> 
> _______________________________________________
> xmlsec mailing list
> xmlsec at aleksey.com
> http://www.aleksey.com/mailman/listinfo/xmlsec
>