ÿþ<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" > <HTML> <HEAD> <title>/umpcFrame : UMPC as a secondary display/virtual remote for Media Center Edition</title> </HEAD> <body style="font-family: Times New Roman"> <TABLE id="Table2" cellSpacing="1" cellPadding="1" width="100%" border="0"> <TR> <TD> <H1>/umpcFrame</H1> <H2>UMPC as a secondary display / virtual remote for MCE</H2> <P><A href="http://www.brains-N-brawn.com/umpcFrame">http://www.brains-N-brawn.com/umpcFrame</A> 9/5/2007 <IMG alt="casey chesnut" src="mailtoCaseyChesnut.png" border="0"></P> </TD> <TD align="right"> <A href="javascript:window.open('http://www.brains-N-brawn.com/Comment/default.aspx?article=umpcframe','','scrollbars=yes,width=550,height=550');void('');"> <FONT size="2">comment(s)</FONT></A>&nbsp; </TD> </TR> </TABLE> <h3> Introduction</h3> <p> er, um ... so i spent $1200 on a UMPC (Ultra-Mobile PC) ... and i need to help justify the purchase. my 'standard line' for justifying purchases is to write code for it. as a developer / power user, the UMPC is not really powerful enough for certain computing scenarios, but its portability and form factor make it useful in other situations. specifically, i want to experiment with the UMPC as a living room device and also for telematics. this article will look at using the UMPC as a secondary display / virtual remote for Media Center Edition (MCE) in a living room setting. </p> <p> the goal was to target functionality that would make me want to keep the UMPC always on and running within easy reach. it should work in both passive and active roles and be operated exclusively with touch. and it should solve problems or enrich the media center experience (e.g. who is that actor?). NOTE the end result is not ready for end users, i rapidly developed this (about 2 weeks in my spare time) as a proof of concept. NOTE havent tested this with an XBox 360 extender, but it could definitely be made to work with one.</p> <p style="text-align: center"> <img src="umpcFrameArch.jpg" />&nbsp;</p> <h3> Alternatives</h3> <p> when thinking about UMPC and MCE, there are a # of different ways to think about integration.</p> <ul> <li>Softsled - software extender, for letting you view your MCE media on the UMPC itself. this app only offers Softsled functionality for picture browsing. for video and music at home, Softsled does not appeal to me ... i'd rather use an XBox 360 as a hardware extender connected to a large TV and a sound system. MS does have a dev kit for creating extenders ... they just dont provide it to independent developers like myself. until MS opens up that dev kit, take a look at <a href="http://www.orb.com/orb/"> www.orb.com</a>.</li> <li>Sideshow - is MS technology for auxiliary displays on PCs and <a href="http://www.ricavision.com/ricavision_webupdate/sideshow_remote.html"> MCE (hardware) remotes</a>. parts of this app could be loosely-defined as a software-based sideshow remote implementation, although it is not using any bits of the Sideshow SDK.&nbsp;</li> <li>MCE local - my UMPC has Vista Ultimate installed, so it can run Media Center locally. this works fine, but just like Softsled, i'd rather view my media on the big screen. plus, you're not likely to have TV tuners connected to your UMPC.</li> <li><a href="http://thegreenbutton.com/forums/thread/183091.aspx">MCE Virtual Remote</a> - is a 3rd party virtual remote for MCE.</li> <li><a href="http://www.maxivista.com/">MaxiVista</a> - is a multi-monitor software that will let you control your MCE machine from your network connected UMPC as a cloned desktop. not sure how video would look over this?</li> <li><a href="http://blogs.msdn.com/coding4fun/archive/2007/05/14/2632164.aspx">Coding4Fun</a> has a great developer article for controlling MCE using Windows Mobile</li> <li>Universal Remotes - there are high end remotes that help to control MCE and all your audio/video equipment. this could be extended to add infrared support (griffin total remote), but it'd be difficult to support all the different components out there; especially to keep that up to date</li> </ul> <h3> Main Screen</h3> <p> the main screen is real simple and just has a list of buttons to get to the other functionality. those features will be explained below.</p> <p style="text-align: center"> <img src="13blogMain.jpg" />&nbsp;</p> <p style="text-align: center"> main screen</p> <h3> Picture Frame</h3> <p> just got back from vacation before i started this, so the first step was to make it a $1200 digital picture frame. it just reads a config file for a list of picture directories and then recursively looks for image files to be displayed. then it just starts cycling through those images at some user set interval. there are touch based controls that lets a user pause shuffling to only show one picture, or to let the user manually page forward and back. my assumption is that this mode will be active most of the time, so this operates independent of MCE, and it will operate as a picture frame even when the MCE machine is off, as long as the pictures are still available locally or on the network. power-wise, the hope is that the picture frame functionality is worth enough that the user disables the screen saver and other power reduction settings so that the UMPC is readily available for other tasks. otherwise it takes too long to wait for the machine to resume from sleep. not implemented yet, but i thought about extending this to also play home videos (muted by default). also need this to support zip archives, because i typically store image files in a single zip.</p> <p style="text-align: center"> <img src="1blogImage.jpg" />&nbsp;</p> <p style="text-align: center"> slideshow of vacation pictures</p> <h3> Auto Channel Surfing</h3> <p> the next feature is auto channel surfing. e.g. the chick gets mad when i endlessly cycle through channels. now /umpcFrame handles it all for me, and the TV can remain on the same channel. this is where it becomes a distributed app. the 2nd piece of the app runs as a background plugin with Media Center. it doesnt have a UI, all it does is host a WCF web service and listen for requests from the UMPC. in this scenario, when a user selects to surf channels, the UMPC sends a request to the plugin. the plugin then finds a TV tuner, tunes to a channel from a list of available channels, grabs a screen capture, and returns the screen capture to the UMPC to display. the UMPC shows the last 6 screen captures at any time. if the user sees a screen capture that interests them, then can select that image and it will switch to a new window that shows previous screen captures for that same channel. then the user can select to tune to that channel and the UMPC will send a request to the plugin to change to that channel. alternately, the user can select to browse the channels, and see a list of the latest screen captures for all channels. NOTE it could be modified to let a user manually cycle through the channels. also, it could be extended to actually stream TV audio/video back to the UMPC. there is a problem here in that the MCE SDK isnt good at sharing tuners. hopefully a future version of MCE will provide better tuner access. also, the MCE SDK doesnt reliably provide states about its current mode of operation. if it did, then the plugin could send a request to the UMPC that the user started watching live TV, and the UMPC could automatically exit picture frame mode and enter channel surfing mode. NOTE at the 11th hour i discovered a bug that keeps this feature from working when MCE is full screen. assumed this was because MCE was running as DirectXExclusive, but setting that to false didnt fix it ... anyway, the bugs still there.</p> <p style="text-align: center"> <img src="2blogTvSurf.jpg" /> </p> <p style="text-align: center"> showing the last 6 channels that have been automatically surfed</p> <p style="text-align: center"> <img src="TvSurfChannel.jpg" /> </p> <p style="text-align: center"> the last 4 screen captures from channel 3</p> <p style="text-align: center"> <img src="10blogTvBrowse.jpg" />&nbsp;</p> <p style="text-align: center"> browsing the last screen capture from all the channels</p> <h3> Web Search</h3> <p> this is real simple, but i'm constantly wanting to search the web for stuff that i see on TV. the usage scenario is that i see an actor and want to see what else they have starred in or if they have any nude pictures available. the problem is i dont want to spend a bunch of time entering text into the UMPC. this just has a textbox that you manually enter the show title or actors name once, and then you can submit that same search to multiple websites without having to re-enter it at each site. NOTE a later screen has a large virtual keyboard. since i think the standard virtual keyboard on the UMPC is a little too small for touch entry, i'd like to hook this up to using my custom virtual keyboard. </p> <p style="text-align: center"> <img src="3blogSearch.jpg" />&nbsp;</p> <p style="text-align: center"> simple web search</p> <h3> Now Playing</h3> <p> generic web search is great, but i also want to do targeted searches off of what is currently playing. so the Now Playing screen attempts to grab metadata for the currently playing video or audio, by calling the MCE plugin. NOTE the MCE API does not provide direct access to metadata for Live or Recorded TV. anyway, if the music has the artist name as metadata, then you can click that button to automatically enter that artists name into the Web Search screen. then you can use the standard Web Search functionality without having to enter any text at all. but the key feature here is duration. the problem is that MCE does not provide random access to media content, you have to fast forward or skip 30 seconds ... which sucks if you want to jump to a certain part of the movie/song. if the audio or video has a known duration, then this screen shows a progress bar on the bottom. so you can just drag that progress bar to where you want it to play, and it will call the plugin to skip to that position in the media. NOTE this could be extended to use round-about methods to get more metadata by using MSAS, dvr-ms metadata, guide data, etc.</p> <p style="text-align: center"> <img src="5blogNowPlaying.jpg" />&nbsp;</p> <p style="text-align: center"> the slider bar is the key; and you can click on a metadata box to redirect to the pre-filled web search</p> <h3> Virtual Remote</h3> <p> of course this needs the obligatory virtual remote, of which i implemented 3 modes. the first mode has the buttons for a standard MCE remote. the second mode has all the navigational pages that MCE understands. the third most is a soft keyboard. the soft keyboard is interesting, because it would allow quick text access within MCE. the problem is, i never got it to fully work right. it works best if you turn caps lock on. only set aside a limited amount of time for this functionality, so i didnt get to debug what the actual problem is. NOTE the standard remote functionality should be updated to support repeat buttons and such.</p> <p style="text-align: center"> <img src="4blogRemote.jpg" /> </p> <p style="text-align: center"> standard virtual remote</p> <p style="text-align: center"> <img src="8blogNavigate.jpg" /> </p> <p style="text-align: center"> navigation to defined MCE pages</p> <p style="text-align: center"> <img src="6blogKeyboard.jpg" />&nbsp;</p> <p style="text-align: center"> soft keyboard for entering text within MCE (not the local UMPC)</p> <h3> Music Playlists</h3> <p> the next feature is for controlling music playback. it doesnt actually playback the music on the device, instead it lets you browse the music and either play it immediately or add it to the queue for playback on MCE. because you're likely to have a much better sound system connected to MCE. this also allows you to control music playback while the MCE machine is on, but the TV is off. NOTE this is currently folder and directory based, just because that was the easiest. it wouldnt be too hard to extend this to support categorizing by artist/album/year/etc. i've had so many problems with the WMP library, that i actually prefer working with the file system directly and wish MCE offered that functionality. this could also be extended to support album art.</p> <p style="text-align: center"> <img src="7blogMusicDirs.jpg" /> </p> <p style="text-align: center"> browsing music directories</p> <p style="text-align: center"> <img src="9blogMusicDialog.jpg" />&nbsp;</p> <p style="text-align: center"> selecting to play a song immediately or add to the queue</p> <h3> TV Guide</h3> <p> the final feature is to show the MCE guide on the UMPC. this allows you to browse the guide without interfering with what is currently playing on the TV screen. so the UMPC calls the MCE plugin, the MCE plugin searches for the relevant programs, serializes them, and returns them to the UMPC to display. if the user clicks on a program in the guide, then they can see more metadata about the program. they can also select the channel and get the program listing for that channel alone. NOTE i dont have this implemented, but this should be extended to let a user select to see 'more info' for a program and then choose to 'record' that show or 'record series'. also to cancel a show that is recording or is scheduled to be recorded.</p> <p style="text-align: center"> <img src="11blogGuideAll.jpg" /> </p> <p style="text-align: center"> viewing the MCE guide on the UMPC</p> <p style="text-align: center"> <img src="12blogGuideChannel.jpg" />&nbsp;</p> <p style="text-align: center"> upcoming programs for NGC</p> <h3 style="text-align: left"> Developers</h3> <p> there is no supported API for getting at the MCE guide data, so i had to write a little code to get at it. ended up creating a typed dataset with relationships to make it easy to move around the data. broke that code out into a simple sample that you can find in the source code section below.</p> <h3> Video</h3> <p> the video below shows its basic features</p> <ul> <li> <a href="http://youtube.com/watch?v=d4iPD2VGtLc" target="_blank"> <img src="umpcFrameThumb.jpg" /> /umpcFrame video</a></li> </ul> <h3> Conclusion</h3> <p> this article detailed a rapid prototype for using a UMPC as a secondary display / virtual remote for MCE. i actually really like the concept of a UMPC living on the coffee room table and being used to enhance a users media experience. my favorite features right now are Auto Channel Surfing and random access of music/video from the Now Playing screen. i listen to 60 minute long mix tracks, so being able to skip around real quick is awesome. being able to keep the TV off is real nice too. granted, it will probably just operate in picture frame mode most of the time. of course there is always the problem of power. right now i've got it constantly plugged into AC power, which is a bit ugly. i'd prefer if it had some sort of quick dock, that positioned it like a picture frame, had power connected to the dock, and allowed me to quickly undock it to use wirelessly. then i could easily dock it to charge when i was done. i think its great that MCE is an open environment that allows for programs like this to be written, but the SDK really needs to be fixed up and extended to support more interesting applications.</p> <h3> Source</h3> <p> here is the C# source code. the umpcFrameWpf solution is .NET 3.0 and runs on the UMPC. the umpcFrameSlaves solution has the umpcFrameMceSlave project which is a background add-in to be run within MCE. it depends on the MCE SDK and DirectShow.NET on SourceForge.</p> <ul> <li><a href="umpcFrame_Source.zip">umpcFrame_Source.zip</a> - contains the WPF app and MCE background add-in</li> <li><a href="GuideSimple.zip">GuideSimple.zip</a> - a stand-alone simple sample app for getting and displaying MCE guide data</li> </ul> <h3> Updates</h3> <p> none planned, but i'll be using this in my living room and will probably update it some. not sure if i'm going to update it enough for an end user release. </p> <h3> Future</h3> <p> think its time for some WPF 3D. later ...</p> </body> </html>