Go to:
Applications -> Preferences -> Sound
There on the Hardware tab. Choose a profile that does NOT contain anything related to digital or IEC. I chose Analog Stereo Duplex. My theory is, that if you go digital there, pulse is blocking the digital output. The sound preferences window seems to be only related to pulse, not alsa at all.
Now open up a terminal window and start:
gstreamer-properties
On the Audio tab, select ALSA - Advanced Linux Sound Architecture, for Default Output.
As Device, choose the digital device, in my case: ALC889A (my sound chip) digital.
I selected ALSA also for Default Input, though I don't think Input has any effect, as long as you don't use a mic or something.
In XBMC, I got both devices for audio and passthrough set to IEC958. And voila :-) I am enjoying happy sound via Alsa in Ubuntu and XBMC, via HDMI and SPDIF, including passthrough in XBMC.
Hope this can help others to avoid audio troubles in Linux.