/spCrypt

comment(s) 

SmartPhone (SP) and CryptoApi with CompactFramework (.NETcf)

http://www.brains-N-brawn.com/spCrypt  10/31/2003 casey chesnut

Introduction

.NETcf does not have the namespace System.Security.Cryptography. Yet we really need it as the world becomes more wireless, and we dont want our data available in the clear for wireless sniffers or network operators, especially with certain wireless security mechanisms being in question (i.e. WEP). Luckily, both the SmartPhone and PocketPC devices come with the CryptoApi. The CryptoApi is an unmanaged API used to call unmanaged cryptographic implementations that exist on the devices as well. This can be done through pInvokes. In this article I will wrap the entire CryptoApi (version 1). As needed, I will then provide higher level wrappers that loosely match the .NET namespace. Finally, I will test for cross platform compatibility between my implementation for CE and the full framework on a desktop.

I looked at the current .NETcf crypto implementations before starting this. The best ones were the Pocket PC Signature Application Sample which has been around for a while, and lately the implementation from a Cryptography chapter in the Microsoft .NET Compact Framework Kick Start (I recommend this book). PPC Signature provides an MD5 hash and uses RC2 symmetric keys for exchanging data. The ManagedCryptoAPI from the Kick Start book provides an MD5 hash, DES and RC4 symmetric encryption for exchanging data, and RSA for asymmetric encryption for exchanging keys. The MD5 hash is compatible with the full framework, but the other ciphers are not out of the box. NOTE I saw a PDC posting today that Whidbey will provide .NETcf with built-in Cryptography support!

SmartPhone

Developed this with the SmartPhone 2003 emulator. 2003 is important, because SP 2002 does not support .NETcf. Also, 2003 has .NETcf with SP1 in ROM. Pocket PC 2003 and Pocket PC Phone Edition 2003 do not have SP1 in ROM, so you could get unexpected behavior. In general though, this code should work on any device that can run the .NETcf and has the CryptoApi available. NOTE I am using the enhanced version of the crypto providers; base providers do not provide implementations for all of the algorithms below

CryptoApi Functions

The 1st order of business was creating the wrapper around the exported functions of the CryptoApi (Crypto.cs) and defining the constants, error codes (Const.cs) and structures (Structs.cs) from wincrypt.h in managed code. Then, I began testing them to make sure they all worked (the ones in RED did not work on the SmartPhone emulator):

As you can see above, all of the calls either work, or are not necessary because their functionality can be obtained from a different call. The pass above was just to see if the calls worked or not. I then moved on to see what interesting things could be accomplished with these calls; because this was my 1st time really working with the CryptoApi

Cryptography

I started targeting the basic crypto services first: Random #'s and working with Providers. Then I moved on to Hashes and Keyed Hashes. Next, was working with Keys and getting access to the raw key data. Finally, Encryption and Signing. The SmartPhone WinForm project in the code has all of these tests. These tests run local on the device and might not be interoperable with the full .NET framework (I make it interoperable later). The actual Crypto wrapper lives in the bNb.Sec project, which is a CE class library and has no dependencies.

Random

This tested multiple ways of generating random numbers, as well as multiple ways of generating GUIDs.

Providers

This lists the available provider types, providers, and ciphers that are available on a device. On a SmartPhone, the Provider Types are:

Each Provider Type above had 2 Providers:

These are the Ciphers and (min, max) key length available for the 2 Enhanced Providers above:

  

Hash

The most popular are MD5 and SHA1, and they both work on the SmartPhone. MD5 is 16 bytes, SHA1 is 20. MD5 is supposed to be faster, and SHA1 is supposed to be stronger. SHA1 is what you will probably use most of the time in your apps. SSL SHAMD5 is used for SSL (probably what Pocket IE uses), it is a combination of MD5 and SHA1 (thus 36 bytes)

Keyed Hash

The keyed hashes most often used are MacTripleDes and HmacSha1. MacTripleDes combines a TripleDES symmetric key with a MAC hash. The MAC hash ends up being 8 bytes. I could not get it to work on the SmartPhone. It would return a byte [] of 8 bytes, but all of the values were zero. If I used DES instead of TripleDES, then it would get a value. An MS employee was helping me in a newsgroup and posted their working C++ code for doing MacTripleDes. I ported that to eVC++, and still got all 0's, so I think this is a bug in the CryptoApi for SmartPhones (maybe for PocketPC too?). A couple of MS employees have recreated this problem and are looking into it. HmacSha1 is a combination of an HMAC and a SHA1 hash. This worked on the SmartPhone with no problems. Originally coded this with using the CALG_HMAC algId. From a newsgroup posting, I ended up having to change this to doing multiple SHA1 hashes after doing some key manipulation. With that change it became more compatible with the full framework. I also tested the CryptHashSessionKey here. I did by getting access to the raw session key data (explained below), manually hashing that key, and making sure those bytes matched the hash obtained by calling CryptHashSessionKey.

Keys

I started out working with symmetric keys (DES, TripleDES, and RC2). Made sure CryptGenKey created random keys each time, and CryptDeriveKey would create the same key each time. Also, attempted to import and export raw session keys. Unluckily, the CryptoApi makes this tedious to accomplish. The rule of thumb is that session keys should never be transferred in the clear, so the CryptoApi forces you to encrypt the session key with an asymmetric key to get the random bytes. Of course, there is a convoluted workaround that lets you get the raw session keys with what is called an exponent-of-one private key. The concept is x * 1 = x, where 1 is the exponent-of-one key. So when you export or import the session key, no bytes are actually changing. When you finally extract a session key in the clear, it will come out being something like 142 bytes. This is odd because session keys are mostly in the range from 8 to 24 bytes. The 142 bytes are a SimpleBlob structure that MS has defined. Its basically header info, the actual key, and then random bytes. Asymmetric keys are easier to import and export. You can just ask for the PublicKeyBlob or PrivateKeyBlob and get access to their plaintext bytes, also in MS defined wrappers.

For symmetric keys, DES is no longer considered secure. RC4 is supposed to be faster than RC2, and RC2 is supposed to be stronger. TripleDES replaces DES and is supposed to be stronger than RC2 and RC4. For asymmetric keys, the PROV_RSA_FULL provides us with RSA which can be used for encrypting and signing. NOTE you should not encrypt and sign with the same key pairs. The PROV_DSS_DH provider types provide DSA. DSA can only be used for signing.

Encryption

The unportable way is to use CryptProtectData with CryptUnprotectData. I'm not exactly sure how CryptProtectData is encrypting the data, but I believe it is tied to a machine. So that if you encrypt with ProtectData on deviceX, then you cannot use CryptUnprotectData to decrypt on deviceY. This seems useful as a means to store keys that need to be reused.

The other methods are CryptEncrypt and CryptDecrypt to be used with the standard symmetric and asymmetric algorithms, which will be portable across devices. Symmetric algorithms are much faster than asymmetric, so the rule of thumb is to use symmetric algorithms to encrypt data with a session key, and then encrypt session keys with pub / priv keys to pass the session key over the wire. For symmetric encryption, you encrypt and decrypt with the same key. For asymmetric encryption you encrypt with somebody else's public key, and they decrypt with their private key.

Signing

This is possible with pub / priv keys. To sign, you use your private key (typically on a hash value), and then that signature can be verified with your public key on another device. To sign with the CryptoApi you use CryptSignHash, and verify with CryptVerifySignature. I made sure that these calls worked and that a signature would not verify if a byte was changed.

Certificates

Most of what we have looked at until now is CryptoApi 1.0, which is basically complete on a SmartPhone. CryptoApi 2.0 introduces more functionality, mainly for dealing with Certificates, with about 1/2 of it existing for SmartPhones. I did not wrap these interfaces as well, and will use the managed System.Security.Cryptography.X509Certificates.X509Certificate class, which does exist in the .NETcf.

 

.NET Interoperability

Now that I knew the basic crypto services worked, I wanted to start testing interoperability with the full framework. To do this I wrapped my code loosely following the System.Security.Cryptography namespaces. All I did was create interfaces, classes, and methods as needed; so it is not as complete. These classes handle the byte reversing and key blob parsing that need to happen between Win32 CryptoApi calls and the real Managed .NET implementation. This same processing would have to happen for the CryptoApi on the desktop, as well as on devices. The code for this wrapper lives in the System.Security.Cryptography namespace (CE class library), and has a dependency on the bNb.Sec project above.

Most of the classes in System.Security.Cryptography ultimately call the CryptoApi for performance in the 1st place (e.g. SHA1CryptoServiceProvider). As opposed to SHA1Managed, which is an entirely managed implementation of the SHA1 hash. Even with the pInvokes, it should be faster to call the CryptoApi then it is to call the pure managed classes. Probably not a big deal for hashes, but definitely for Asymmetric encryption and signing. Also, some of the ciphers just do not exist in the CryptoApi for SmartPhones at all: Rijndael, SHA256, SHA384, SHA512

My method for testing involved the same .NETcf WinForm client running on the SmartPhone emulator (used above), along with a WebService running on a desktop using the full .NET Framework. Then the client would do its version of the crypto call and pass its results to the WebService. The WebService would verify those results on its end, and then perform the same operation, passing back the results to the CE client, which would then verify those results. (e.g. encrypt on device and send to WS, WS decrypts and checks that bytes, WS then encrypts and responds, device decrypts and checks that bytes match). The SmartPhone unit test project is called spCrypt, and the ASP.NET WebService interop test harness is wsCyrpto.

RNGCryptoServiceProvider

This one is simple. All it does is wrap CryptGenRandom. It looks random enough, so I didnt make a WS call to make sure the bytes didn't match :)

ToBase64Transform and FromBase64Transform

Just use Convert.ToBase64String and Convert.FromBase64String instead

MD5CryptoServiceProvider and SHA1CryptoServiceProvider

These are just simple hashes. The hash obtained on CE ended up being the same on the WS, so they match with no changes needed. NOTE that SHA256, SHA384, SHA512 dont exist in the CE CryptoApi

MACTripleDES

This is the Keyed Hash that I could not get to work above on CE at all, so I could not test it. MS is looking into it as a bug.

HMACSHA1

This keyed hash worked on the desktop, and the WS got the same hash value byte for byte.

PasswordDeriveBytes

This is used to derive session keys. You provide it a password and the name of a key cipher and hash, and it should derive the same session key time every time. In this manner you only have to remember your password, and dont have to store the session key. When I did an RC2 key and a SHA1 hash, the keys matched on the device and the desktop. When I did a DES key with a MD5 hash, some of the bytes were off by 1. I think this is due to 'salting'. I tried turning off the salt but could not get the derived keys to match exactly. In general, PasswordDeriveBytes is probably not compatible outside of the CryptoApi (regardless of desktop or device).

DES / TripleDESCryptoServiceProvider and RC2CryptoServiceProvider

I dont like the frameworks object model for ciphering with session keys, so I did not reproduce it. Instead I created the classes but made EncryptValue() and DecryptValue() methods like on the RSA class (as opposed to CryptoStreams). I tested DES, RC2, and TripleDES and was able to encrypt and decrypt to and from both the desktop and device. The only trick was the keys were stored in reverse by the CryptoApi. So keys I sent to the desktop had to be flipped, as well as keys being returned from the desktop before importing on the device. NOTE that Rijndael does not exist on the CE Crypto API, nor does RC4 or RC5 exist on the desktop.

RSAParameters and DSAParameters

The RSA and DSA classes will export and import using these classes, as well as to and from Xml. This implementation will handle both, so you can export and import both RSA and DSA keys between the desktop and the device. There is a bunch of bit flipping involved to do this.

RSACryptoServiceProvider

This class does a lot. 1st I made sure that it could encrypt and decrypt data between the WS. Along with the key flippings, the resulting ciphered data also had to be reversed. After that, I could encrypt on the device and decrypt on the WS, and vice versa.

DSACryptoServiceProvider

Similar to RSA above, except DSA can only be used for signing. This one had a different blob format then RSA which required me to export the key twice to get all info for private keys. Also, the full framework exports a derived parameter 'J' which I recreated by pilfering a Mono BigInteger class. Also the resulting signature had to be byte flipped twice (in sections) to get it to work with the full framework.

RSAPKCS1KeyExchangeFormatter and RSAPKCS1KeyExchangeDeformatter

These are just thin wrappers over RSA, that do encryption and decryption of session keys. I tested by creating a session key on the device, using my RSAPKCS1KeyExchangeFormatter , and sending the result to the WS to use its RSAPKCS1KeyExchangeDeformatter and create a session key from it; and vice versa. The keys exchanged matched byte for byte. NOTE the full framework has OAEP (instead of PKCS1) versions of these as well. I do not know if the CE CryptoApi can do OAEP?

RSAPKCS1SignatureFormatter and RSAPKCS1SignatureDeformatter

Same as the above, but instead of encryption and decryption, it calls SignHash and VerifyHash on the RSACryptoServiceProvider. This checked out on both the desktop and device as well.

DSASignatureFormatter and DSASignatureDeformatter

Same as the above SignatureFormatter, but for DSA instead of RSA.

Diffie-Hellman

The full framework does not have Diffie-Hellman, but the PROV_DSS_DH provider type on CE seems to. This is used for key agreement, as opposed to key exchange with RSA above. In RSAPKCS1KeyExchange, the session key was encrypted and sent over the wire. In key agreement, the session key is derived by both parties and never transferred over the wire. So that if I have my private key and your public key, and you have your private key and my public key, then we can both derive the same session key to exchange info, without actually having to exchange the session key so that somebody could compromise it. Pretty cool! NOTE DH is compromised, if either of the private keys is acquired.

Key Management

The Crypto Api treats symmetric (session) and asymmetric (pub/priv key pair) differently. Session keys only live in memory, so if you need to reuse one you have to handle that yourself. One option might be using CryptProtectData and CryptUnprotectData to save it to disk. For key pairs, each named container stores 2 key pairs, 1 for exchanging (e.g. encrypting) and 1 for signing. For PROV_RSA_FULL, you get 2 RSA key pairs because it can be used to encrypt and sign. For PROV_DSS_DH, you get DSA for signing and Diffie-Hellman keys for key exchange. Also, these key pairs are stored in the registry, so when you are importing and exporting keys, you have to be careful not to step on your own keys. To handle this, all session keys use their own named container, and the key pairs for that container will end up being 'exponent-of-one' keys so that the session keys can be exported in the clear. Therefore, those pub/priv keys should never be used for exchanging or signing. For RSA and DSA, my implementation uses the users key by default. I do this for speed, because it takes some time to generate a key each time. There is a static property that can be set to make it gen a key each time. Also, when importing keys, it uses a different container so as not to destroy the users key. Sometimes I get an error BAD_KEY_STATE when using the user key (mostly with PROV_DSS_DH provider). To get around this, I have a ResetKeyState method in the Key class. This would destroy your key pairs and create new ones, so you might want to export them to Xml and save them locally using ProtectData and then import them later on. To complicate things even more, if you are sending a message that needs to be signed and encrypted, then you should use different key pairs. So for RSA (which can do both), encryption uses the EXCHANGE key pair, while signing uses SIGNATURE key pair. Better yet, use RSA for encryption and DSA for signing.

Performance

SmartPhones have processors as low as 132 megahertz, so we have to think about speed. When starting out to code this, I was either going to pInvoke the CryptoApi, or port the Mono implementation that is all managed. My assumption was that the cost to pInvoke the CryptoApi was going to be less than doing the ciphers in all managed code. Also, the different algorithms will perform differently. MD5 is supposed to be faster than SHA1. RC4 is supposed to be much faster than RC2 (although RC4 doesnt exist on the full framework). Symmetric algorithms are much faster than asymmetric algorithms, so the rule of thumb is to encrypt the data with symmetric, then encrypt the symmetric key (which is small) with an asymmetric algorithm. Next, dont sign big chunks of data; instead take a hash of the data and just sign the hash. Finally, if you are encrypting a large chunk of data with a symmetric key, then you will want to display progress to the user. My wrapper does not currently allow for the data to be encrypted in chunks, but it could be extended to do so because the CryptoApi does. If performance is a real big issue, an eVC wrapper could be written which exposes methods consisting of multiple CryptoApi calls to reduce the number of pInvokes required for each cipher.

Conclusion

So we now have interoperability between the desktop and the device with almost all of the System.Security.Cryptography namespace. MACTripleDES seems to be broken on CE, so use HMACSHA1 instead. PasswordDeriveBytes is partially compatible with RC2 / SHA1, and DES / MD5 is close (except for salting); although this might not be compatible on other platforms outside of the CryptoApi. Finally, the CryptoApi providers on the SmartPhone offer no way to do the hashes (SHA 256/384/512) nor Rijndael (symmetric) encryption. 

Books

I'm a crypto wannabe, so I referred to these books during writing this

Source

Update

1/25/04 here is a bug fix per Jason Weiss (Thanks Jason!) look for his Cryptography book  sometime this quarter:

I was looking over your code and noticed that you had documented in the code that your description wasn't working properly in the CryptoUnprotectData method (Line 123 of Cipher.cs). Essentially, the API documents it as a ppsz, which is a pointer to a pointer of a null-terminated string. Passing in StringBuffer directly works great if you are referencing an lpsz, but a ppsz needs an IntPtr by ref. If you change the decl to pass a ref to an IntPtr, like this:

private static extern bool CryptUnprotectDataCe(
   ref CRYPTOAPI_BLOB pDataIn, ref IntPtr ppszDataDescr, IntPtr pOptionalEntropy, IntPtr pvReserved, 
   IntPtr pPromptStruct, uint dwFlags, ref CRYPTOAPI_BLOB pDataOut); 
instead of your StringBuilder, and then in the method body do this:
//Assuming a max size of 99 characters in the description null terminating character
IntPtr ppszDescription = CryptMemAlloc(100);
bool retVal = CryptUnprotectData(ref blobIn, ref ppszDescription, optEntropy, reserved, prompt, (uint) flags, ref dataOut);
desc = Marshal.PtrToStringUni(ppszDescription);

Future

I wrote this with the intention of asking for volunteers to help move it into OpenNetCF.org workgroup to continue work on it. With the MS announcement that they will be adding Cryptography to .NETcf, I'm less interested, but could be persuaded otherwise? Also, I have a follow up article (why I wrote this library in the 1st place) that will be released within a couple of weeks ...