"Fantastic" may be overstating it. I've been running Raspbmc for about 15 months now, and although the video rendering is very good, the XMBC UI is a bit laggy and unresponsive (although there seems to have been an improvement recently). I've also used it as an SFTP client and to run a few light cron jobs (rock solid there).
Given the price, I am pleased with it, but I wouldn't recommend it as a media center for the non-tech savvy.
Agreed. I wish that RPi has some kind of cheap prebuild of a mediaplayer. If would be GREAT for non-techies if it just had a real power on/off switch, IR sensor & a basic remote that just works (probably asking for too much there).
I reckon you could fix the lag of the GUI if someone implemented Raspbmc/OpenELEC with Wayland instead of X-server. Not much has to change for this to be possible. Only the power switch & IR sensor would have to be apart of it. Remote could potentially be third party and WBMC would a man hours issue.
If you're willing to spend quite a bit of time configuring XBMC it runs smoothly. I use openelec with the theme 'bello', any animation heavy themes will kill it straight away.