/umpcFrame

UMPC as a secondary display / virtual remote for MCE

http://www.brains-N-brawn.com/umpcFrame 9/5/2007 casey chesnut

comment(s) 

Introduction

er, um ... so i spent $1200 on a UMPC (Ultra-Mobile PC) ... and i need to help justify the purchase. my 'standard line' for justifying purchases is to write code for it. as a developer / power user, the UMPC is not really powerful enough for certain computing scenarios, but its portability and form factor make it useful in other situations. specifically, i want to experiment with the UMPC as a living room device and also for telematics. this article will look at using the UMPC as a secondary display / virtual remote for Media Center Edition (MCE) in a living room setting.

the goal was to target functionality that would make me want to keep the UMPC always on and running within easy reach. it should work in both passive and active roles and be operated exclusively with touch. and it should solve problems or enrich the media center experience (e.g. who is that actor?). NOTE the end result is not ready for end users, i rapidly developed this (about 2 weeks in my spare time) as a proof of concept. NOTE havent tested this with an XBox 360 extender, but it could definitely be made to work with one.

 

Alternatives

when thinking about UMPC and MCE, there are a # of different ways to think about integration.

Main Screen

the main screen is real simple and just has a list of buttons to get to the other functionality. those features will be explained below.

 

main screen

Picture Frame

just got back from vacation before i started this, so the first step was to make it a $1200 digital picture frame. it just reads a config file for a list of picture directories and then recursively looks for image files to be displayed. then it just starts cycling through those images at some user set interval. there are touch based controls that lets a user pause shuffling to only show one picture, or to let the user manually page forward and back. my assumption is that this mode will be active most of the time, so this operates independent of MCE, and it will operate as a picture frame even when the MCE machine is off, as long as the pictures are still available locally or on the network. power-wise, the hope is that the picture frame functionality is worth enough that the user disables the screen saver and other power reduction settings so that the UMPC is readily available for other tasks. otherwise it takes too long to wait for the machine to resume from sleep. not implemented yet, but i thought about extending this to also play home videos (muted by default). also need this to support zip archives, because i typically store image files in a single zip.

 

slideshow of vacation pictures

Auto Channel Surfing

the next feature is auto channel surfing. e.g. the chick gets mad when i endlessly cycle through channels. now /umpcFrame handles it all for me, and the TV can remain on the same channel. this is where it becomes a distributed app. the 2nd piece of the app runs as a background plugin with Media Center. it doesnt have a UI, all it does is host a WCF web service and listen for requests from the UMPC. in this scenario, when a user selects to surf channels, the UMPC sends a request to the plugin. the plugin then finds a TV tuner, tunes to a channel from a list of available channels, grabs a screen capture, and returns the screen capture to the UMPC to display. the UMPC shows the last 6 screen captures at any time. if the user sees a screen capture that interests them, then can select that image and it will switch to a new window that shows previous screen captures for that same channel. then the user can select to tune to that channel and the UMPC will send a request to the plugin to change to that channel. alternately, the user can select to browse the channels, and see a list of the latest screen captures for all channels. NOTE it could be modified to let a user manually cycle through the channels. also, it could be extended to actually stream TV audio/video back to the UMPC. there is a problem here in that the MCE SDK isnt good at sharing tuners. hopefully a future version of MCE will provide better tuner access. also, the MCE SDK doesnt reliably provide states about its current mode of operation. if it did, then the plugin could send a request to the UMPC that the user started watching live TV, and the UMPC could automatically exit picture frame mode and enter channel surfing mode. NOTE at the 11th hour i discovered a bug that keeps this feature from working when MCE is full screen. assumed this was because MCE was running as DirectXExclusive, but setting that to false didnt fix it ... anyway, the bugs still there.

showing the last 6 channels that have been automatically surfed

the last 4 screen captures from channel 3

 

browsing the last screen capture from all the channels

Web Search

this is real simple, but i'm constantly wanting to search the web for stuff that i see on TV. the usage scenario is that i see an actor and want to see what else they have starred in or if they have any nude pictures available. the problem is i dont want to spend a bunch of time entering text into the UMPC. this just has a textbox that you manually enter the show title or actors name once, and then you can submit that same search to multiple websites without having to re-enter it at each site. NOTE a later screen has a large virtual keyboard. since i think the standard virtual keyboard on the UMPC is a little too small for touch entry, i'd like to hook this up to using my custom virtual keyboard.

 

simple web search

Now Playing

generic web search is great, but i also want to do targeted searches off of what is currently playing. so the Now Playing screen attempts to grab metadata for the currently playing video or audio, by calling the MCE plugin. NOTE the MCE API does not provide direct access to metadata for Live or Recorded TV. anyway, if the music has the artist name as metadata, then you can click that button to automatically enter that artists name into the Web Search screen. then you can use the standard Web Search functionality without having to enter any text at all. but the key feature here is duration. the problem is that MCE does not provide random access to media content, you have to fast forward or skip 30 seconds ... which sucks if you want to jump to a certain part of the movie/song. if the audio or video has a known duration, then this screen shows a progress bar on the bottom. so you can just drag that progress bar to where you want it to play, and it will call the plugin to skip to that position in the media. NOTE this could be extended to use round-about methods to get more metadata by using MSAS, dvr-ms metadata, guide data, etc.

 

the slider bar is the key; and you can click on a metadata box to redirect to the pre-filled web search

Virtual Remote

of course this needs the obligatory virtual remote, of which i implemented 3 modes. the first mode has the buttons for a standard MCE remote. the second mode has all the navigational pages that MCE understands. the third most is a soft keyboard. the soft keyboard is interesting, because it would allow quick text access within MCE. the problem is, i never got it to fully work right. it works best if you turn caps lock on. only set aside a limited amount of time for this functionality, so i didnt get to debug what the actual problem is. NOTE the standard remote functionality should be updated to support repeat buttons and such.

standard virtual remote

navigation to defined MCE pages

 

soft keyboard for entering text within MCE (not the local UMPC)

Music Playlists

the next feature is for controlling music playback. it doesnt actually playback the music on the device, instead it lets you browse the music and either play it immediately or add it to the queue for playback on MCE. because you're likely to have a much better sound system connected to MCE. this also allows you to control music playback while the MCE machine is on, but the TV is off. NOTE this is currently folder and directory based, just because that was the easiest. it wouldnt be too hard to extend this to support categorizing by artist/album/year/etc. i've had so many problems with the WMP library, that i actually prefer working with the file system directly and wish MCE offered that functionality. this could also be extended to support album art.

browsing music directories

 

selecting to play a song immediately or add to the queue

TV Guide

the final feature is to show the MCE guide on the UMPC. this allows you to browse the guide without interfering with what is currently playing on the TV screen. so the UMPC calls the MCE plugin, the MCE plugin searches for the relevant programs, serializes them, and returns them to the UMPC to display. if the user clicks on a program in the guide, then they can see more metadata about the program. they can also select the channel and get the program listing for that channel alone. NOTE i dont have this implemented, but this should be extended to let a user select to see 'more info' for a program and then choose to 'record' that show or 'record series'. also to cancel a show that is recording or is scheduled to be recorded.

viewing the MCE guide on the UMPC

 

upcoming programs for NGC

Developers

there is no supported API for getting at the MCE guide data, so i had to write a little code to get at it. ended up creating a typed dataset with relationships to make it easy to move around the data. broke that code out into a simple sample that you can find in the source code section below.

Video

the video below shows its basic features

Conclusion

this article detailed a rapid prototype for using a UMPC as a secondary display / virtual remote for MCE. i actually really like the concept of a UMPC living on the coffee room table and being used to enhance a users media experience. my favorite features right now are Auto Channel Surfing and random access of music/video from the Now Playing screen. i listen to 60 minute long mix tracks, so being able to skip around real quick is awesome. being able to keep the TV off is real nice too. granted, it will probably just operate in picture frame mode most of the time. of course there is always the problem of power. right now i've got it constantly plugged into AC power, which is a bit ugly. i'd prefer if it had some sort of quick dock, that positioned it like a picture frame, had power connected to the dock, and allowed me to quickly undock it to use wirelessly. then i could easily dock it to charge when i was done. i think its great that MCE is an open environment that allows for programs like this to be written, but the SDK really needs to be fixed up and extended to support more interesting applications.

Source

here is the C# source code. the umpcFrameWpf solution is .NET 3.0 and runs on the UMPC. the umpcFrameSlaves solution has the umpcFrameMceSlave project which is a background add-in to be run within MCE. it depends on the MCE SDK and DirectShow.NET on SourceForge.

Updates

none planned, but i'll be using this in my living room and will probably update it some. not sure if i'm going to update it enough for an end user release.

Future

think its time for some WPF 3D. later ...