/spWse

comment(s) 

SmartPhone (SP) and WebService Enhancements (WSE) with .NETcf

http://www.brains-N-brawn.com/spWse 11/17/2003 casey chesnut

Introduction

So i had taken my 1st cut at WSE from the Compact Framework (.NETcf) many months ago ... (/cfWSE). Since then, i have gotten to do a lot more Web Service (WS) work using the .NETcf. So I felt I was ready for a 2nd attempt to raise the bar ... although this article is really the 3rd in a long running series. I could not continue the /cfWSE article because .NETcf was lacking the System.Security.Cryptography namespace; so that is why I wrote the follow up /spCrypt article. Now I can continue where I left off. My motivation is 2-fold: (1) because WSE does not exist for .NETcf. (2) give me an excuse to learn some more of the Global Xml Architecture (GXA) specifications. This time, client-side development will be with .NETcf using the SP 2003 emulator. Server-side will mostly use the WSE 2.0 implementation (currently a tech preview), along with some of the WSE 1.0 sample web services.

 

SmartPhone

Developed this with the SmartPhone 2003 emulator. 2003 is important, because SP 2002 does not support .NETcf. Also, 2003 has .NETcf with SP1 in ROM. Pocket PC 2003 and Pocket PC Phone Edition 2003 do not have SP1 in ROM, so you could get unexpected behavior. In general though, this code should work on any device that can run the .NETcf. You must also take into account what level of cryptography and algorithms the device / emulator provides (below).

WSE1

WSE has been released and supported by MS for a while. The WS specs it supports are WS-Addressing, DIME, and WS-Security.

WSE2

This is the latest release of WSE that has been a tech preview for a while. It had some breaking changes between it and version 1. One such change was moving to WS-Addressing instead of WS-Routing. It still supports DIME and WS-Security. It also adds support for WS-Trust, WS-Policy and Ws-SecureConversation.

Service Pack 1 (SP1) Rant

For SmartPhone, .NETcf and SP1 are in the ROM! For Pocket PC 2003, .NETcf is in ROM, but not SP1 ... so install it. The problem is the download is just a bunch of zip files, that you have to manually copy around. It explicitly states no new interface changes, but there are some methods that I want to overload now which just became available in SP1; i can deploy the runtime to support them but I cant use VS .NET to build to that. To get VS .NET to build with that overload for CE / PPC, I have to unzip the files that were deployed and then manually rename those files to compile against, or build against an assembly meant for SP (see below). My hope is that .NETcf SP installation will integrate with VS .NET in the future.

GetWebRequest / GetWebResponse

These are the methods on HttpWebClientProtocol that can now be overridden with SP1. This is useful to set HttpHeaders, KeepAlive, and such on the underlying HttpWebRequest and HttpWebResponse used by the autogenerated web reference. For a SP project, you can to it right off the bat. For CE or PPC you have to update the System.Web.Services.dll that VS .NET builds against. What I had to do was copy the assembly that SmartPhone builds against into the CE directory, and then I could override that method on PPC and CE apps as well. You also have to make sure (at least) SP1 is on the emulator or device that you deploy to.

SP - C:\Program Files\Microsoft Visual Studio .NET 2003\CompactFrameworkSDK\
	v1.0.5000\Windows CE\Smartphone\System.Web.Services.dll
CE / PPC - C:\Program Files\Microsoft Visual Studio .NET 2003\CompactFrameworkSDK\
	v1.0.5000\Windows CE\System.Web.Services.dll

Session

ASP.NET handles session by using cookies. On the full .NET framework (.NETfx) you can access the CookieCollection to add/remove cookies, but not on .NETcf. So if you wanted to do stateful WS calls your options were cookieless sessions (required WS change), or handle all the SoapMessaging by using the lower level HttpWebRequest/Response classes. By hooking GetWebRequest/Response, you can add the proper cookie headers directly to the HttpWebRequest without changing the service, and still have the convenience of using a web reference proxy. A better way would be per keith ba's WS book, in which Session information is passed using a SoapHeader, and it is not reliant upon a protocol. I do not know of a specification that exists for this?

Service Pack 2 (SP2) Rant

Same as above for SP1, plus you dont get an update for SmartPhone. SP2 has some Web Service fixes and performance improvements... which I obviously want. My hope is that SP2 fixes some of the bugs I will detail below. Maybe it will eventually be deployed by the phone carriers, since they can update the ROM? NOTE per an MS newsgroup posting, SP2 will be in the next SmartPhone SDK release ... whenever that is :)

SoapHeaders

One of the SP2 fixes is supposed to handle how null SoapHeaders are handled. I think the behavior is that if you have a null SoapHeader as InOut on the .NETcf, then it will send the element as xsi:nil= "true", while the full framework will not send the SoapHeader at all. Another has to do with namespace handling. Something to the effect of on the .NETcf I had to explicitly declare a Namespace on a child element to be the same as the parent, while this re-declaration did not have to be done on the full framework. There was a similar issue with XmlAttributes and how they had to be declared more explicitly. Related to this, there is an SP2 fix for XmlTextReader and namespace handling which comes in handy as well.

WS-Utility / Timestamp

I had implemented this previously. All I did was extend it to more fully support the specification. I ran it against the WSE2 Timestamp sample, and it will probably work for the WSE1 Timestamp sample as well. One thing to note is that the SP emulator displayed the same time as my desktop, but its default time zone was (GMT London, Dublin) so all of my calls would fail until I changed the TimeZone and the Time. It was annoying to do this manually, so I wrote a little pInvoke class to handle updating the TimeZoneInformation bias for me. This gets called every time my test app loads, so you would have to modify it to fit your own time zone. Also, if you leave the Emulator running for a while (e.g. about an hour), the time will get out of synch with the desktop by a number of minutes. So if your TimeStamp expiration time is about 5 minutes, then you will start getting TimeStamp expiration SoapFaults. Just restart the emulator and it will synch again.

//request
<Timestamp xmlns="http://schemas.xmlsoap.org/ws/2002/07/utility">
  <Created>2003-11-07T23:48:21Z</Created> 
  <Expires>2003-11-07T23:53:21Z</Expires> 
</Timestamp>

//response
<wsu:Timestamp>
  <wsu:Created>2003-11-07T23:49:34Z</wsu:Created> 
  <wsu:Expires>2003-11-07T23:50:34Z</wsu:Expires> 
</wsu:Timestamp>

Web Reference Wrappers

Instead of changing autogeneated web reference proxies (to add SoapHeaders and such), I always subclass them. This lets me update the web references frequently as the interface changes and not worry about losing any Attributes I have added to that code. The problem is the way I overloaded the WebReference invoke of the WebMethod using the 'new' keyword does not seem to work on the full framework. It fails with the 'methodName' cannot be reflected exception. I think the proxy code generator should be changed to add the virtual keyword. NOTE when I tested on the full framework, my clients typically did not use WSE. WSE clients inherit from Microsoft.Web.Services.WebServicesClientProtocol so they might exhibit different behavior.

{any} and  {@any}

The X in Xml stands for eXtensible. This flows into the specifications themselves. Alot of times you will see {any} and {@any} which represent any XmlElement and any XmlAttribute can show up as valid nodes. Used to be all over HailStorm. If a web service declares them, then your client will be generated with them as well. Great for the .NETfx, but they have issues on .NETcf. On .NETfx, if it is an 'in' SoapHeader with an XmlAnyElement or XmlAnyAttribute, then the WS would get called if you left them as null. Sometimes you would get this behavior on .NETcf. For the TimestampHeader, I would either get a NullReferenceException or an IndexOutOfRangeException and no stream would be sent out for the WS request. I would get similar results if I actually set the values before trying to make the WS call. For retrieving 'out' SoapHeaders from a web service call, I can get XmlAnyElement and XmlAnyAttributes on the SoapHeader itself. If that SoapHeader element contained a sub element that also contained an XmlAnyElement, that would work; but if it contained an XmlAnyAttribute, then it would fail with an exception something like: 'Text' is an invalid node type. Line 1, position 392. So I would typically decorate the SoapHeaders accordingly with XmlAnyElement or XmlAnyAttributes and then comment them out as things broke. NOTE that XmlAnyElement and XmlAnyAttribute are different than UnknownSoapHeaders. XmlAnyElement and XmlAnyAttribute would be child nodes of a known SoapHeader.

//request
<AnyShIn xmlns="http://tempuri.org/" strAttribute="strAttrib">
  <strElement>strElem</strElement> 
  <anyShSub strAttribute="strAttrib">
    <strElement>strElem</strElement> 
  </anyShSub>
</AnyShIn>

//response
<AnyShOut strAttribute="strAttrib" myAnyAttrib="myAnyAttribValue" xmlns="http://tempuri.org/">
  <strElement>strElem</strElement> 
  <anyShSub strAttribute="strAttrib">
    <strElement>strElem</strElement> 
    <myAnyElem myAnyAttrib="myAnyAttribValue" xmlns="" /> 
  </anyShSub>
  <myAnyElem myAnyAttrib="myAnyAttribValue" xmlns="" /> 
</AnyShOut>

Tracing

When testing the above, I had to do alot more debugging then I preferred. This involved a lot of sniffing what was actually being sent over the wire. My preferred method is to use the Soap Toolkit Trace Utility. You can redirect to it at a certain port, and it will listen for incoming requests and then redirect to another endpoint. The really nice thing is that it can parse DIME messages as well. Occasionally my app would exception because of 'no end of entity mark', this seems to be caused by a parsing bug in the Soap Toolkit Trace Utility? If I would make the same WS call again immediately after, it would work fine. Also, I used the de facto TraceExtension SoapExtension example. Only changes it needed were to remove the SoapServerMessage usage and to write out to a filepath that exists on the SP. Then the problem is being able to view the actually file because there is no file explorer on SP. One of the managed code samples that comes with the SP SDK is a file explorer (CompactNav). With a little code addition (below), it can open .txt and .xml files in Pocket IE. Another option might be to use the ActiveSync-Emulator powertoy that has come out, which might let you browse to the file, although it has been way too finicky for me to use. Finally, WSE has some nice tracing options built-in for the server-side if you control that endpoint as well.

else if (fullPath[depth].ToLower().EndsWith(".txt") || 
				fullPath[depth].ToLower().EndsWith(".xml"))
{
	//add to bottom of CompactNav.listView1_ItemActivate
	//after if (fullPath[depth].ToLower().EndsWith(".exe"))
	ProcessInfo pi = new ProcessInfo();
	// launch the exe using our pinvoked CreateProcess call
	CreateProcess(@"\Windows\iexplore.exe", fullPath[depth], pi);
	// back up since we didn't actually enter the directory
	--depth;
}

WS-Routing

This specification is basically dead. WS 2.0 uses WS-Addressing instead. I did go ahead and update this SoapHeader to better support the spec for existing WSE 1.0 apps that are already live. Tested it against the WSE1 Routing sample, which ends up routing the call to the SumService web service. NOTE the referralCache.config change below

//request
<wsrp:path soap:actor="http://schemas.xmlsoap.org/soap/actor/next" soap:mustUnderstand="1" 
    xmlns:wsrp="http://schemas.xmlsoap.org/rp">
  <wsrp:action>http://microsoft.com/wse/samples/SumService/AddInt</wsrp:action>
  <wsrp:to>http://localhost/wsequickstart/router/sumservice.asmx</wsrp:to>
  <wsrp:id>uuid:a435db9a-f8a1-453a-a75e-8ca8839b3863</wsrp:id>
</wsrp:path>

WS-Referral

This spec. all happens on the server, to handle the actual routing of messages, so we do not have to worry about it from the client side. It is worth noting that you will have to change the referral.config file to develop with a SmartDevice. This is because SmartDevices cannot call web services with 'localhost' domains, and have to use machine name or ip address. The WSE Routing samples use localhost, so have to be modified to the ip address or machine name that you use from your device. If you have already called the WSE Routing sample, then referralCache.config will be locked, so you will have to iisreset to unlock it and make the changes. NOTE that capitalization might be an issue. Also if you are using a Soap tracing mechanism that depends on port redirection, then you will have to make changes for that as well.

WS-Addressing

I dont really like this spec. because it has a number of SoapHeaders that are scattered about and not contained by something like an <Addressing/> parent element. I did implement it as SoapHeaders, as well as in a SoapExtension (below). Tested it against the WSE2 Routing sample, which calls the BaseStockService web service. NOTE the referralCache.config change above

//request
<wsa:Action>http://stockservice.contoso.com/wse/samples/2003/06/StockQuoteRequest</wsa:Action> 
<wsa:From>
  <wsa:Address>http://schemas.xmlsoap.org/ws/2003/03/addressing/role/anonymous</wsa:Address> 
</wsa:From>
<wsa:MessageID>uuid:140eaeb5-bb6d-58b1-8f31-ed56619e353b</wsa:MessageID> 
<wsa:To>http://notebook:8080/TimeStampService/TimeStamp.asmx</wsa:To> 

//response
<wsa:From>
  <wsa:Address>http://schemas.xmlsoap.org/ws/2003/03/addressing/role/anonymous</wsa:Address> 
</wsa:From>
<wsa:MessageID>uuid:de8f245e-cb33-4366-bfb5-70f9445f6b17</wsa:MessageID> 

SoapExtension

Manually adding all those SoapHeaders for Timestamp and Addressing got annoying. With the WSE, routing and timestamp info is sent from the client without you having to specify anything. I wanted this same sort of automagic behavior, so decided to wrap the SoapHeaders with a SoapExtension. In the SoapExtension, on BeforeSerialize, you can add SoapHeaders to a Headers collection. On .NETfx, these will get serialized and sent out just fine, but not on .NETcf. On .NETcf, you have to hook AfterSerialize, load the stream into an XmlDocument, add the proper XmlElements to the doc, and then handle the stream properly. This sucks, because you lose all the typing. Also, when calling XmlDocument.Load, .NETcf closes the underlying stream, while .NETfx does not. A related trick (per Alex Feinman) to this is to add a generic SoapFault to the web ref proxy, and then you can add a typed SoapHeader and it will be sent appropriately.

The other problem with SoapExtensions is that you have to register them using Attributes on .NETcf. I wish there was a programmatic way, so that my application logic could add them dynamically. Christian Weyer came up with a slick way to register them on .NETfx by reflecting to the app.config way of registering files; there should be a similar way to do the same through the client-side attribute method although i have not dug into it.

Finally, there is another problem when stacking SoapExtensions. On the full framework, you can stack multiple SoapExtensionAttributes on top of the client proxy web method, and each SoapExtension will run in order based on whether the response is incoming or outgoing. This fails on the .NETcf because the stream will not get passed properly from SoapExtension to SoapExtension. You can get around this by just having one SoapExtension, and call a handler for each type of logic in the proper order. I really hope the problems with SoapExtensions get fixed for Whidbey. Mainly so that most of the code can be moved from doing Xml manipulation in AfterSerialize and BeforeDeserialize to working with typed objects in BeforeSerialize and AfterDeserialize. I would have preferred that programming model instead :(

Proxy Authentication

When calling an external WS, I tried it behind a proxy that required authentication. Initially, this was a problem to get the WSDL. The VS.NET 'add web reference' was not able to handle it, although it looks like wsdl.exe can. To get around this I used IE to browse to the WSDL, then saved it locally and reference the locally saved file in VS.NET to generate the proxy. Once the proxy was created, then it became a problem when calling the actual WS. On the .NETcf this was handled by catching a WebException, and then inserting the proper credentials. On .NETfx I would have done this by adding NetworkCredentials to the GlobalProxySelection.Select. Although that class exists on .NETcf it did not work for me and I had to add the NetworkCredentials to the webRefProxy.Proxy property instead. A UI would have to be supplied to let a user enter credentials, so hopefully this will not happen to SPs that are wirelessly connected because user input is a pain

DIME / WS-Attachments

My 1st version of this only was able to send one attachment out. In DIME this is actually 2 DimeRecords, the 1st being the SoapMessage, and the 2nd being the actual Attachment. It was also ugly in that you had to use the HttpWebRequest class directly and build up the SoapRequest manually. First, I extended this by making it support multiple attachments going out. Next, I updated it to support receiving multiple files in a Dime Response. Finally, I rehooked the SoapExtensions to work with the modified WSE interface and port to run on .NETcf. This makes it work with the web reference proxies so you dont have to manually get the SoapRequest. I doubt that chunking would work; which is used to break up a file among multiple DimeRecords in a single DimeMessage. Tested this against the WSE2 Attachments sample and it should also work for WSE1 DIME-enabled web services. The .xml files below show the DIME header parsed by the Soap Toolkit Trace Utility. The .txt files show the raw HTTP streams. I modified the sample to return DIME as well, so neither request nor response is your typical SOAP message

dimeReq.xml / dimeReq.txt / dimeRes.xml / dimeRes.txt

Cryptograpic Algorithms

I actually wrote the previous parts of this article (Addressing and DIME) about a month ago. Then I started tackling WS-Security, and had to do a major segue. Reason being is that 2 big aspects of WS-Security are Xml-Encryption and Xml-Signature, both of which depend heavily on cryptographic algorithms. Since the .NETcf does not have the System.Security.Cryptography namespace at all, I spent almost a month filling in those missing pieces. The result is my /spCrypt article.

The System.Security.Cryptography wrapping ended up being rather successful. Both the Xml-Enc and Xml-Sig specifications name a number of algorithms, and whether they are required / recommended or optional. For Required, we are missing AES Block Encryption. In a recent MSDN magazine, a C# AES implementation was provided that will compile in .NETcf. It works, except I could not get it to work with data larger than 16 bytes. Don't know if I was calling it wrong? Regardless, TripleDes is the other required Symmetric algorithm, and WSE uses it by default. For key transport we are missing RSA-OAEP, although we have RSA-PKCS1 which is used by default. For Recommended algs, we are only missing a SHA256 hash, but we do have the required SHA1 hash. For Optional algs, the smartPhone does have Diffie-Hellman for key agreement, although the full .NETfx does not support it. Finally, MacTripleDes is not working for the device but it is not needed for either Xml-Enc or Xml-Sig. Also, we dont have Rijndael symmetric (related to AES), but its not needed for WS-Security either. The result is the CryptoApi wrapping gave us access to just about everything we will need to continue.

WoTe

Write once, Test everywhere !?!? ... I knew C# looked familiar :) The encryption to support WSE entails having an enhanced PROV_RSA_FULL provider found on most Pocket PC PCs, including PPC 2002, but not on the 2002 emulator. For PPC 2000, this can be added by installing the enhanced encryption pack. Although my Sys.Sec.Crypt library supports the DSA algorithm with the PROV_DSS_DH provider found on SP 2003 and PPC 2003, it is not yet necessary for making WSE2 calls. I extended the .NETcf crypto bits to look for if CryptoApi, the Enhanced Provider, and DSA are available.

I have tested the /spCrypt libraries on the PPC 2003 and CE emulators and got working results. Same goes for this articles unit tests with SP1 installed. Initial testing on my PPC 2002 device (Audiovox Maestro) showed some missing functions with Guid and Random # generation that can be overcome by using different methods. There was an unnecessary Crypto call being made (CryptProtectData) that resulted in a MissingMethodException. One way of doing HmacSha1 would throw a BAD_ALGID when destroying the key, but HMAC can also be done with multiple SHA1 calls. And RC2 symmetric encryption did not interop with the full framework, although it does on the emulators for SP 2003, PPC 2003, and CE .NET. So the variables are Device, OS, CryptoApi, and Service Pack ... and this should all be tested to get the codebase to the lowest common denominator. To help with this, I wrote a WoTe class to find Device, OS, and CLR info.

My current rule of thumb is to write all class libs as Windows CE projects. Then put a thin SmartPhone UI over that for testing. For kicks, I make PPC and CE WinForms project and run the SP UI in those environments as well. It ports with no changes, the only thing is that the menu operation is funny with a stylus. For even more kicks I will run those apps directly on a desktop computer. Hopefully device and desktop .resx files will be made compatible in Whidbey to make this easier. VS .NET could stand to do a better job of importing existing 'Web References' as well. Below are screen shots of the SmartPhone WSE unit test app running on a PPC 2003 emulator, CE .NET emulator, and on a desktop computer respectively.

  

WS-Security

This is the lowest level security specification for web services. There are a number of other security specifications which build upon this one (e.g. WS-Policy, WS-Trust, WS-SecureConversation). This one is the core, so it has to be built 1st. WS-Security is primarily used for 3 different scenarios: Authentication, Confidentiality, and Integrity. I will tackle these in this same order.

Authentication (Security Tokens)

This is actually the reason I began writing this article. 1st the Microsoft.com web service went public. It is basically a 'hello world' web service meaning it does not expose much functionality yet; but what made it interesting is that it required a UsernameToken SoapHeader to access the service. 2nd a Perl implementation of UsernameToken came out before a .NET one did, other than the WSE. Who cares if you are on the full framework, because you have the WSE, but .NETcf was out of luck. So I threw together a quick SoapHeader implementation and threw it out to the newsgroups. That was the decision point when I decided to attempt WSE for .NETcf again. NOTE you will have to supply your own username and password to access the service.

<Security xmlns="http://schemas.xmlsoap.org/ws/2002/07/secext" soap:mustUnderstand="1">
  <UsernameToken d4p1:Id="SecurityToken-2506db16-f1c8-2129-7f36-13b576716003" 
      xmlns:d4p1="http://schemas.xmlsoap.org/ws/2002/07/utility">
    <Username>Admin</Username> 
    <Password Type="wsse:PasswordDigest">ABC...=</Password> 
    <Nonce>yHZfWUJgSUMiUXcEdsPce2YXK6I=</Nonce> 
    <d4p1:Created>2003-11-08T00:05:19Z</d4p1:Created> 
  </UsernameToken>
</Security>

UsernameToken - Had already got it working against Microsoft.com, so I knew it was at least partially working. Next, I attempted to call the WSE2 sample. Before calling it, I modified the sample app to make sure it did not have to pass a Signature, and it worked without one. When I tried to call it, it kept throwing a SoapFault with 'The Soap Header Security was not understood'. Looking at the trace of the message from the WSE2 client, the 'wsse' namespace version was /2002/12/ instead of /2002/07/, so I upgraded mine and it worked. To test it more, I ran it against the WSE1 UsernameSigning sample, and it worked fine. I did have to modify the WS IsValid() method, so that it only looked for a UsernameToken SecurityToken, and not a Signature SecurityElement.

<Security xmlns="http://schemas.xmlsoap.org/ws/2002/12/secext" soap:mustUnderstand="1">
   ...
</Security>

BinaryToken - this is another type of Security Token that can be passed. For this one, all I did was read in an X509v3 certificate that was stored as a file on the device. Then I got the raw data and base64 encoded it. Passed it to an echo web service I wrote that just read the certificate and spit back the name. It worked with WSE2. Make sure you only pass around public certificates using this method.

<Security xmlns="http://schemas.xmlsoap.org/ws/2002/12/secext" soap:mustUnderstand="1"> 
  <BinarySecurityToken d4p1:Id="SecurityToken-acbfa8b0-ad6e-9164-3c25-4838ee2c2a68" 
    ValueType="wsse:X509v3" EncodingType="wsse:Base64Binary" 
    xmlns:d4p1="http://schemas.xmlsoap.org/ws/2002/07/utility"> 
      ABC...== 
  </BinarySecurityToken> 
</Security>

Other Security options include Kerberos and custom (binary & xml) tokens. I'd been reading about Kerberos and wanted to attempt it, but it requires my environment to have a domain, which I dont have the time to setup :(. My code would have to be extended to support these.

Confidentiality (Xml-Encryption)

The next level of difficulty is XmlEncryption. Started out with SymmetricEncryption from WSE1, because that sample does not exist for WSE2. Initially, the sample just encrypted the request, and the response came back in the clear. For the request, the SoapHeader needed a <Security\SecurityTokenReference> to be added along with the <EncryptedData> in the SoapBody. Made a SoapExtension to do this. The client application has to set some static objects on the SoapExtension that tells it how it wants to encrypt and decrypt, and the SoapExtension handles the rest. At 1st, the web service could not decrypt my messages, and was throwing back an XmlException. To test that my System.Security.Cryptography was really compatible, I grabbed the encrypted message from the WSE1 client trace, and decrypted it. This revealed that the messages were getting IV's appended to them, per the XENC spec when I looked it up. Added this logic to my code, and I got a successful response from the WebService. Then I extended the WS to send back an encrypted response as well, and updated the XmlEncryption SoapExtension do decrypt it.

xmlEncSymmReq.xml / xmlEncSymmRes.xml

Then I went to the WSE2 ResponseEncryption sample. In this one, the request sends a X509 certificate to the WS as a BinarySecurityToken in the SoapHeader; and the response returns an EncryptedKey in the SoapHeader, with EncryptedData in the SoapBody. For X509 Certificates, see the section below. The X509 cert has the public key that the WS uses to encrypt the response. The service creates a TripleDes session key to encrypt the SoapBody. That session key is then encrypted with the RSA public key from the certificate. On the client side, it discovers the EncryptedKey 1st, and uses the corresponding RSA private key to decrypt. That session key is then loaded into a TripleDesCryptoServiceProvider to decrypt the actual SoapBody. Works great. I did have to mod the web service to not require an XmlSignature at this time, and just look for the BinarySecurityToken directly.

xmlEncRes.xml

The final one is the WSE2 AsymmetricEncryption sample. The ResponseEncryption sample also uses Asymmetric encryption, but only the Response. This is the same, except it is for building the request, while the response comes back in plain text. One other difference is that instead of sending the public key as a BinarySecurityToken, it sends a KeyIdentifier. So the client generates a session key. That session key is encrypted with the servers public key and added as an EncryptedKey to the SoapHeader. That session key is also used to encrypt the SoapBody and replace it with an EncryptedData element.

xmlEncAsymmReq.xml

Right now it supports symmetric and asymmetric encryption. EncryptedKeys and EncryptedData. Named keys, X509 certificates, and known KeyIdentifiers. Encrypting the entire message, and also just a portion of the message using my LameXpath implementation (does anybody have a good Xpath imp. for .NETcf?). It would have to be extended to support referenced keys and arbitrary data, as well as multiple EncryptedKey and EncryptedData sections in a single message.

X.509v3 Certificates

From playing around with the WSDK (WSE1 alpha), I still had a bad taste in my mouth from X509 ... it was a real pain. WSE2 is better in that they have more documentation and even provide some sample certificates and tools to use. It was a little tricky to get them setup correctly on the smartPhone. On the desktop, you can use the Certificate MMC plugin to import the certificates. For the client you should import the Client Private.pfx and the Server Public.cer. The .pfx files have both public/private key pairs, and the .cer files only have public keys. On the server side you should import the Server Private.pfx. I also exported the Client Private.pfx without the private key to Client Public.cer, and imported the public key for the web service as well. The Client Public.cer also comes in handy on the SmartPhone, because the System.Security.Cryptography.X509Certificate class can read them directly. NOTE it could only read .cer files exported as DER, but not B64 unless they were decoded 1st. So the SmartPhone has the client public key, the server public key, but not the client private key. This was tricky. To get the client private key, I modified the desktop  KeyPal code posted to a security newsgroup my Michel Gallant. It lets you view the crypto containers, and export keys. When looking at this, it showed the certificate crypto containers. Modified the code to export both the public and private keys. So I exported the Client Private.pfx certificate to Client Private.pvk. This blob just happens to be the same structure when you call CryptExportKey using the low-level CyrptoApi; which I just happened to wrap in the /spCrypt article. So I can read that file with the private key on the device, and import the key to do decryption and signing on the device.

Another missing piece for Certificates it the KeyIdentifier. Not exactly sure how this info is acquired yet, but I know you can manually obtain it using the WseCertificate tool that comes with WSE2. Since the samples I am targeting only require the KeyIdentifier to be sent from the client to the WS, then I will just hardcode that KeyId on the client for now.

Finally, for encrypting messages and verifying signatures, we will use the Server Public.cer certificate. So we need to know how to get the .cer file into CyrptoApi format. Luckily, Michel (again) posted this code about an hour before I needed it: DecodeCertKey. Modified that code to accept an X509Certificate and spit out the exponent and modulus to be used in an RSAParams class.

Integrity (Xml-Signature)

I was worried about getting this piece to work because of XmlCanonicalization. Until I found a Mono implementation. Instead of building signatures 1st, I started with validating them. The request trace from the WSE2 X509Signing sample validated immediately. Then I made sure that it was really working by changing the body of the message, and then the SignedInfo element, and it did not verify for both of those instances. The UsernameSigning References validated the same, but I had problems figuring out how the signature was being done. The SignatureMethod is an HMACSHA1, which requires a key. It is documented how to generate the key using the P_SHA1 algorithm from TLS, except there is no mention of the label being 'WS-Security' and the key size being 24 bytes. Fumbling around with that for a couple days and I was finally able to verify signatures from WSE. If I manually modified the signature or references, then verification would fail as expected too. NOTE this was the only point I had to resort to Anakrino / Reflector to figure out what was going on. All other code was written according the the WS specifications and viewing traces between the WSE2 clients and services. Also, UsernameToken signing is up for review because of interoperability issues between implementations.

unsReq.xml / x509Sign.xml

Next, I went about building the signatures. Started with the WSE2 X509Signing Web Service 1st. It accepted the signature from my client, and validated it with the public certificate that was sent with it. Then I went on to the UsernameSigning sample and got it to work using the key generation method explained above. This implementation emulates the full WSE and signs the Body, Creates / Expires of Timestamp, and Action / From / MessageId / To of WS-Addressing. It supports all of these by default, but also lets you specify explicitly what you want (or dont want) to be signed. Also, its untested, but should support signing with DSA as well as RSA; although I dont think WSE supports DSA yet?

xmlSigUser.xml / xmlSigX509.xml

Xml-Enc and Xml-Sig

I had previously modified the WSE2 ResponseEncryption sample to turn off signing. Went back and turned that on, so that it would send a signature out, and receive an ecrypted response. Created a SoapExtension that would add the Timestamp and Addressing headers, as well as XmlEncryption and XmlSignature if necessary. Called the WS, and it worked, doing XmlSignature for the request and XmlEncryption for the response for the same call.

xmlEncSigReq.xml / xmlEncSigRes.xml

The more observant reader would see the scam above. Its not really Xml-Enc AND Xml-Sig, the previous test was Xml-Enc OR Xml-Sig. From all the reading I did, I kept running across sections detailing the problems with doing both. e.g. do you sign 1st and then encrypt, or do you encrypt 1st and then sign the EncryptedData? So I figured I had better test this. The 1st thing I did was extend the ResponseEncryption client sample. I set it up so that it would take 2 tokens. The servers public key to encrypt the message, and the clients private key to sign it. Running the client, it made a request that was both signed and encrypted, and the WS was able to handle it and return the results. Saving off that request, I made sure that my bits could validate it. Ended up it needed to be decrypted 1st and then the signature had to be checked. Then I updated the SoapExtension to reflect this, and made sure on outgoing requests, that it would sign 1st and then encrypt. Ran it against the WSE2 sample, and it failed. Ended up having to position the EncryptedData element before the Signature element, and then it worked correctly. Then I further extended the sample so that the WS would return a signature as well, and my client was able to validate it. So the traces below show WS-Utility Timestamp, WS-Addressing, and WS-Security (XmlEncryption and XmlSignature) used at the same time! NOTE be careful that you do not sign and encrypt with the same key pair, as this lends itself to a crypto attack.

encSigBothReq.xml / encSigBothRes.xml

Programming Model

The following code shows how to use the bNb.Ws class lib for doing both XmlEnc and XmlSig (for the request and response traces immediately above). The first step you have to do is add a web reference to the WSE-enabled web service as you normally would. Then you have to decorate it with the WsExtension SoapExtension which executes the handlers for Timestamp, Addressing, XmlEncryption, XmlSignature, etc... In your code you have to deal with the X509Certificates and Key pairs used for signing and encrypting. Then you have to set static objects on the handlers which specify how to encrypt/decrypt and/or sign/validate the requests and responses. Finally you call the WS using the web reference as you normally would, and the SoapExtension does the WSE processing. NOTE this could certainly be shaped into the same client-side programming model provided by the WSE.

//get my public key to send in request
//WS will use this to verify signature and encrypt response
...steps removed...
X509Certificate cert = new X509Certificate(rawKey);

//get my private key to sign outgoing request
//also used to decrypt response from WS
...steps removed...
RSACryptoServiceProvider rsaCsp = new RSACryptoServiceProvider();
rsaCsp.FromXmlString(privKey.ToXmlString(true));

//setup object to sign request and validate response
bNb.Ws.XmlSigObject xso = new bNb.Ws.XmlSigObject();
xso.BinSecTok = new bNb.Ws.BinarySecurityToken(cert, bNb.Ws.EncodingType.Base64Binary);
xso.AsymmAlg = rsaCsp;
bNb.Ws.XmlSigHandler.SigObj = xso;

//setup object to decrypt response
bNb.Ws.XmlEncObject xdo = new bNb.Ws.XmlEncObject();
xdo.RSACSP = rsaCsp; //for header
xdo.SymmAlg = new TripleDESCryptoServiceProvider(); //for body
bNb.Ws.XmlEncHandler.DecObj = xdo;

//get server public key to encrypt session key
//random session key is used to encrypt SoapBody
...steps removed...
RSACryptoServiceProvider rsaCspServ = new RSACryptoServiceProvider();
rsaCspServ.ImportParameters(rsaParam);

//setup object to encrypt request
bNb.Ws.XmlEncObject xeo = new bNb.Ws.XmlEncObject();
xeo.EncKey = new bNb.Ws.EncryptedKey();
xeo.SymmAlg = new TripleDESCryptoServiceProvider(); //random session key
xeo.RSACSP = rsaCspServ; //to encrypt session key
xeo.KeyId = "bBwPfItvKp3b6TNDq+14qs58VJQ="; //servers key id
bNb.Ws.XmlEncHandler.EncObj = xeo;

//call web service
string [] symbols = {"FABRIKAM", "CONSTOSO"};
RespEncServ.StockQuote [] sqa = respEncProxy.StockQuoteRequest(symbols);

Performance

There is definitely a performance hit with WS-Security. Especially since this is on a device with limited resources, and we are doing cryptography along with Xml processing. But as the Practical Cryptography book says 'we have enough fast unsecure systems out there'. To mitigate this, it is a must to do asynchronous calls as to not freeze the UI; as well as making chunky calls instead of being chatty. Service Pack 2 (not for SmartPhone yet) will be faster too. Also, I had to change programming models midway into this. Started out trying to work with objects as SoapHeaders, but ended up switching to XmlDocument processing. At that point I put performance to the side, and just wanted it to work. This could be sped up by switching to XmlReader / XmlWriter. Next, there is a noted processing delay (server side) when working with the X509Certificates provided with WSE2. Supposedly legit certificate will perform faster. It would be interesting to see what the cost is for using WS-Security, as compared to running without it. Also, to see what the difference is between WS-Security and SSL/TLS ...

PPC 2002 Device

Of course, I dont actually have a smartPhone, so my current concept of its speed / slowness could be entirely wrong :( But I do have a beat up PPC 2002. First, I got the /spCrypt articles unit test to run on my device (above). NOTE none of the limitations with the crypto stuff on PPC2002 have to do with the crypto that is used for WSE. Then I made this articles unit test work on my device. For the most complicated calls (both signing and encryption), it took just over 5 seconds for the call to complete. NOTE my PPC has the SP2 beta running on it. SP1 should work too, because that is what SmartPhone uses, but might be slower. Dont know if it will run without either of the .NETcf service packs installed?

Conclusion

This should give you a great start for calling the next generation web services with the .NETcf. It provides the fundamentals of WS-Routing / Addressing, DIME, and WS-Security (including Xml-Encryption and Xml-Signature).

I explicitly did not attempt some of the WSE2 samples. Avoided any that used Microsoft.Web.Services.Messaging, including: HttpSyncStockService, TcpAsyncStockService, and TcpSyncStockService. Also did not do ContentBasedRouting because all the client does is read a referralCache.config file to see what Url the router is at. CustomFilters happens on the server, but could be done client-side with SoapExtensions. PipelineHosting was out of scope.

Along with Michel for help with Certificates, I need to thank Sebastien Pouliot, whom helped me past many Cryptography and WS-Security sticking points from his contributions to Mono.

Books

These are the books I referred to while writing this

Source

Updates

12/30/2003 - some bug fixes and extensions to the DIME lib

Future

I might keep working on this to start doing the higher level WS-Security stack and new GXA WS specs as they are implemented in future WSE drops. Hopefully Indigo will not forget about mobile devices ... Regardless, it is good that MS is now starting to promote mobile web services with Vodafone