/noMadMap

comment(s) 

Compact .NET (Mobile) + MapPoint .NET (Web Service)

http://www.brains-N-brawn.com/noMadMap 5/27/2002 casey chesnut

Background

If you have been browsing the MS mapPoint or compactFramework newsgroups, then you could have guessed that I was putting this article together. Have made my niche to take 2 bleeding edge technologies, think of some silly idea to use them together, and then document my endeavors. Sometimes will tackle 2 technologies with absolutely no experience in either, but this time I thought it would be cake, cause had kicked around both of these previously; and wanted to reinforce my knowledge ... or lack thereof, as we shall soon find out :( It's bleak until the very end :)

Idea

So I got the brand new CompactFramework Beta 1 release, and the 1st thing I did was crack it open and consume a HelloWorld Web Service. Worked Great! Then, checked out the 4 reasons the techPreview sucked (listed above), and they had fixed all of those. It ran decent, there was design time support, exceptions had more info, and web service consumption was taking place. Hell yeah! So then I say, lets do something for real. At the same time, was pissed off from every stinking book I was reading having an intro to Web Services chapter in them. So decided to tie it in with a legit Web Service. So my listing of full on legit Web Services turned up: MapPoint and Google. Had just integrated Google into my /pass application for intelligent agent action, so did not really want to do work with it again. Also, had recent memories of being totally lost and new in Pittsburgh, so MapPoint was a shoe in. Also, still needed to call the Routing service that MapPoint exposes. So the one thing I did not like about MapPoint was the chatty nature of the service (e.g. If you have a zip, you have to find the lat/long of that zip, then find surrounding entities of interest, and then get a map rendered with that info - 3 round trips). Bet that they end up extending the service to expose Batch type APIs. Until then, part of the beauty of the CompactFramework was that it would let me keep state, so my app could be less chatty. So that is the reasoning behind the idea. Actually, I had a much cooler idea for integrating this with .NET My Services, but a complete lack of vision has sent that effort underground.

Day One (5/1) - Hurry up to wait

Created a new Smart Device Application project - Pocket PC flavor. Added MapPoint .NET staging web reference. Then needed some place to store my credentials for accessing MapPoint. Saw that System.Configuration was there, but was missing appSettings, so could not read from appName.exe.config, and was missing my beloved dynamicProperties :). Had not used a resource file in .NET yet, so threw them in there and got that working (more on configuration later). As far as UI, decided on a TabControl with multiple pages:

  1. Addresses - page for adding/deleting and generating maps for a particular address
    • when saving, if a network is available it will get the lat/long by calling the Find service and then store that info to reduce round trips
    • maps that were generated would be saved for disconnected use
  2. Directions - provides two drop downs for selecting a to/from address and then getting directions between the 2 from the Route service
    • text directions would be stored for when network was not available to save trees
    • if the address did not have a lat/long yet, then it would get it and save it off now
    • would get the generated map showing the route overlayed and save it off
  3. Maps - this would be a simple repository for choosing a previously saved map to view
    • the mapView is a seperate form to reclaim maximum screen space
  4. Settings - had to throw this in so that when released, people could easily change their MapPoint credentials

Needed some place to store data. My options were serialized DataSet or SQL CE. Thought SQL CE was overkill since would not need to sink with a parent DB, nor need to store alot of data on the Pocket PC. Was really upset that Typed DataSets are not supported, but did the pseudo typing workaround by generating an XSD and then tying that in to a generic DataSet. NOTE somebody on the newsgroup was able to generate a typed DataSet for the CF (CompactFramework) by stripping out alot of the serialation stuff. Saw what he was doing, and decided not to go down that route because it would have been difficult to maintain, and I wasnt dead set on my XML Schema being correct yet. If the ICodeParser was available with CodeDom, then would have made an app to programmatically gen CF-compliant typed DataSets. Building off a previous effort to carry XSD enum facet info along with Typed DataSets: http://www.brains-N-brawn.com/strongerTDS  Had to use a FileStream and XmlTextReader to read the XSD and XML into the generic DataSets. Looks like the interface is there to read from a filePath, but it was not working for me. Also had to remove null values from my XML file because it kept throwing System.FormatExceptions. After getting that base stuff set up, was ready to call the Web Service. Modified my pre-existing helper class for calling the MapPoint Find service. Looked like the only code change was that the .PreAuthenticate() method was no longer available. Tested to make sure I could pull that out of my other codeBase and that it would still work as well, and sure enough it did. Started, calling the FindMethod and got System.InvalidOperationException/Could not load resource assembly. Suck! This is when the pain began ...

Probably had some other design decisions here, but this was along time ago ... as you shall see ... so I forget what they were.

3+ Weeks Pass - Bored

During this time ... off and on ... I peruse the newsgroups, send off some whining emails asking for help, beat my head against the wall, and try every brute force methodology I can to get it to work ... to no avail ... pretty much end up shelving it because of CF Tech Preview flashbacks.

The very minimal code sent to the MS guys for calling MapPoint .NET from CF can be found here: dkMapSamp.zip It has the enum to allow the FindAddress call to work, but not the GetEmptyWebProxy() method, so when you hit the 'Map' button, you might get this: output.JPG The odd thing is that it works for them right out of the box!

Day Two (5/25) - More fun/pain

Start putting together the calls to MapPoint .NET

Day Three (5/26) - Sniff sniff

Yesterday afternoon stunk, so its time to start sniffing. One of the MS guys had recommended a couple diff packet sniffers. MS NetMon was my favorite for Win2K Server, but I'm on WinXP Pro now, and dont think you can install it on it. At least, could not find it. Tried out the ones he recommended and thought Ethereal was the best ... 'Sniffing the glue that holds the Internet together'. So busted it out, and started making requests. Saved 3 interactions each from both the CompactFramework and full .NET Framework to compare the results. You should be able to open these files in Ethereal if you want

NOTE would have much rather created a SoapExtension class to just write the request and responses off to file, instead of sniffing, but that class does not exist in the CF. Really dont think this is going to fly as people fumble around with their own security mechanisms and such while the Web Services standards solidify. Not that you can do heavy encryption on a 200mhz PPC, but their are light encryption algorithms out there

NOTE Each request was made with .PreAuthenticate not being set to true for PocketPC and the Desktop. GetEmptyWebProxy was ON for PocketPC calls unless specified and GetEmptyWebProxy was OFF for Desktop calls unless specified 

The txt saves are basically unreadable, even in Ethereal. But Ethereal has this option, where if you click on an HTTP line, then you can right click and say Follow TCP Stream. This gives you a much prettier dump of the associated HTTP Header, SOAP Request, HTTP Header, and hopefully SOAP Response. Did not do this for every request/response ... traced the larger file to note all the request and associated responses, and then broke out the pertinent ones that would allow me to do some pattern matching to see if I could figure out what was going on

NOTE the SOAP request is not even sent from the CF if one of the enums (LatLong or Direction/CalculatedRouteRepresentation) is not set. Initial guess is that the SOAP serializer fails beforehand with the enum value 0 is unknown. It is sent fine on the full .NET Framework, need to check out the differences between those requests.

NOTE So much for minimizing round trips! Have no clue why I kept getting so many 401 Unauthorized messages. Possibly because I took off the .PreAuthenticate()? Not like I sniff packets everyday (no comments please), but if that is what the internet is running on ... suck. Gathered the PocketPC data by using the PPC Emulator. The Desktop data is from Internet Explorer. Both were run off of my development machine, which is behind a Linksys switch connected through DSL. My actual PCC either connects through ActiveSync over USB to my dev machine, or directly to the switch with a Linksys CompactFlash 802.11b card

Right click-Save for the #'d txt files. You can click on the txt files in the tables to open them in the browser

  1. _Ppc-Find.FindAddress-Render.GetMap.txt
  2. _Ppc-Find.FindAddress-Render.GetMap (without GetEmptyWebProxy).txt
  3. _Ppc-Find.FindAddress-Route.CalculateSimpleRoute(2)-Render.GetRouteMap.txt
    • This is from pressing the Route button on my Directions tabPage. It had to find the lat/long for both addresses with the LatLong enum. Then it calls to get that Route twice, once with the Directions enum and then with CalculatedRouteRepresentation. Finally it calls GetRouteMap. Does not work because it exceptions about not having MapView within the Route results.
    • POST FindService 401 Unauthorized 
      POST FindService 200 OK
      POST FindService 200 OK
      POST RouteService 401 Unauthorized PPC-Route.CalculateSimpleRoute-401.txt
      POST RouteService 200 OK PPC-Route.CalculateSimpleRoute-200.Direction.txt
      POST RouteService 200 OK PPC-Route.CalculateSimpleRoute-200.Calculated.txt
      POST RenderService 401 Unauthorized
      POST RenderService 401 Unauthorized
      POST RenderService 401 Unauthorized
      POST RenderService 401 Unauthorized
      POST RenderService 500 InternalServerError PPC-Render.GetRouteMap-500.txt
    • Internal Server Error is serialized to a full on SoapException
  4. _Desk-Render.GetMap.txt
  5. _Desk-FindAddress(2)-Route.CalculateSimpleRoute(2)-Render.GetRouteMap.txt
  6. _Desk-FindAddress(2)-Route.CalculateSimpleRoute-Render.GetRouteMap (with GetEmptyWebProxy).txt

Ok, my brain is near fried putting this together and I have not even analyzed it yet.

Headed to the gym at my apartment complex to relax the brains- and work on the -brawn. Being that I solve my hardest problems at the gym, realized that I still had some more variables; such as it might be my network connection OR a bad installation of the SDE. So I'm going to throw this over to my notebook, and walk over to Kinkos to see what's up. Going to walk there to continue my interrupted workout ... and just so I dont get lost :)

An hour later, and initial results look like it is not my network connection NOR installation. Got 401 Unauthorized errors from the desktop and emulator, as well as the Route service did not successfully work being called from the CF. Same thing, only minimal text was returned, nor was the total distance or driving time available.

Right click-Save for the these txt files

For little success, wrote simple file logic to support archiving of maps for the mapPage. Would be done if I could just get text directions and get a map with route overlay to generate.

 

Have knocked out a bunch of variables, but I need to do a recap because this is getting confusing:

Now, I'll start walking that list, plus more, to see if I can figure out what in the world is going on. Will be looking at the txt files above of SOAP request/responses to divine this info

Hacks within hacks (Dune perversion)

This is how stubborn; err, umm ... persistent I am. From the above, the only real issue I dont have a workaround for is not getting the full results from the Route.ResultMask.Directions SOAP response deserialized properly into my runtime object. Specifically, the text to be saved off and displayed, total distance/driving time, and the View to pass off to Render.GetRouteMap. Yeah, yeah, yeah ... it's beta; but I dont want to wait, and might learn some more stuff by pushing the issue.

  1. Try new'ing the View object before calling Render.GetRouteMap ... think I tried this, and just forgot to document, but not sure ... stupid stupid
    • new was not enough, but it complained about a CenterPoint after that. So calculated that with the idea from step 3 below and it rendered this map!
    • Sweet success! The picture sucks, but when I set the CenterPoint, noticed a bounding rectangle property, so will set that next.
    • My work here is done ... sweet! Actually, it still has the following bug that I cannot fix:
    • See how the path goes off the screen to the left. Since my hack is creating the bounding rectangle off of the to/from locations, then I cannot handle that. NOTE how the image is rotated, more on this later
  2. Direct access to the SOAP Response before deserialization (same for the SOAP request before serialization because then could set the enums to null to make chunky calls). This would allow me to get the text directions from the response, and also create the appropriate request for Render.GetRouteMap ... preferred hack
    • So all I have left to do is get the txt directions.
    • Checking the generated proxy, it comes down to this:
      • object[] results = this.Invoke("CalculateSimpleRoute", new object[] { geoWaypoints, resultMask});
      • return ((Route)(results[0]));
    • By doing a quickwatch on the results array, see that I have already lost the info that is in the SOAP result, and not in the cast ... so without a SOAP extension, do not think it can be hooked with the CF
  3. Figure out the centerPoint between the from/to address. Gen a CenterPoint map from that, and then overlay the CalculatedRouteRepresentation, if those bits properly render to an image ...  a hack to be proud of
    • saved off the bits from the  CalculatedRouteRepresentation ... not sure what it is, but it is not a gif, so this would not have worked; but the centerPoint idea spawned the idea that ultimately got 1 above to work 
    • Here is a link to those bits: nmm_route.gif GCRR? NOTE i really mean that it is not a gif, so you cant view it as one
  4. The Render.GetMap returns the image, as well as a View element. That might be useful for a perversion of 3 above
    • unnecessary at this point

Outstanding Issues

Hoping somebody from MS will help me address these

Image Rotation

The display on PPC is 320x240. With actual viewing area for apps being 300x240. Since space is so critical, wanted to squeeze as much out of it as I could. To do this, decided to ask for a differently sized image from the MapPoint service and then rotate it on the client. i.e. for paths that travel mostly east to west (based on bounding rectangle), will want to ask for a 240x300 image from the web service and then rotate it to 300x240 to display on the device. The user can just rotate the device in their hand to read it. The result being:

Since the CF Bitmap implementation did not have its own .Rotate() method, I wrote my own by doing pixel flipping. This ended up being SLOW! 2 minutes on the emulator per image and 45 seconds on my actual PPC device. My 1st idea was to have the server to the rotation, but MapPoint .NET does not currently offer that. 2nd idea was to just add a progress bar ... which I did

3rd idea, which I will revisit, is instead of doing the bit flipping between 2 bitmap objects; should attempt to use the lower level System.Drawing.Graphics object. Until the, this delay only happens for paths that are longer horizontally then vertically

Image Overlay

The image rotation added this extra requirement, because I could easily see getting confused which wasy was North. So this just takes a little graphic and overlays it on the map to display which way is North. Although, you can do this on the server using the service, this is not processor intensive and I wanted to do it client-side for kicks. The result being (my Kinko's trip):

And for a map that has been rotated, it is pointing left.

Until you flip your PocketPC. NOTE this map is the rotated version of a map above, the one that I got to render after the bounding rectangle hack. This gives you an idea of how much space was reclaimed

Future

This is stuff I still want to do

Source

My full source is here, minus my password! You will have to enter your own MapPoint credentials to actually run it

Full C# Source Code: noMadMapSrc.zip - updated 5/31

Full Ethereal Packet Sniffs: noMadMapEth.zip

Updates

5/28/2002 - fixed some bugs, will zip up that code later. converted article to MS Reader eBook .lit format so you can read on your PPC  noMadMap.lit

5/31/2002 - finished off most of everything i wanted to do with this: including image rotation and image overlays. the new version of the source is posted. got my speech .NET beta in the mail, so i probably will not touch this anymore.

7/4/2002 - Just saw that the CF team have released a MapPoint .NET sample on GotDotNet.com: http://www.gotdotnet.com/team/netcf/mappoint/default.aspx It includes an update to the CF beta that fixes the WebService bugs that I had to hack around above ... "The MapPoint .NET sample contains both a Visual Studio .NET project targeting the .NET Compact Framework, and an update to the Web Service components of the .NET Compact Framework."

7/18/2002 - installed the update mentioned above, and it works great. here is the code to retrieve the textual output that I could not retrieve previously, as well as a screenshot of the resulting output.

dirBox.AppendText("dist: " + r.Direction.Directions[0].Distance.ToString("##.##"));
dirBox.AppendText("\r\n");
dirBox.AppendText("time: " + r.Direction.DrivingTime.ToString("##.##"));
dirBox.AppendText("\r\n");
foreach(SegmentDirection sd in r.Direction.Directions)
{
	foreach(DirectionPrimitive dp in sd.Direction)
	{
		dirBox.AppendText(dp.Instruction + " " + dp.Distance.ToString("##.##"));
		dirBox.AppendText("\r\n");
	}
}

This image is one pulled directly from the file system of my PPC. Before I could do image rotations and overlays. No clue why Houston and Toronto have boxes around them?