These instructions are for Ubuntu 11.10 on which XFCE was installed using either the Ubuntu Software Center or sudo apt-get on the command line.
The instructions may be the same for other versions, but no guarantee is made.
When XFCE is installed in Ubuntu 11.10, it installs several additional packages and libraries. Not all these packages and libraries are uninstalled when you uninstall XFCE. Several packages (like Thunar) do not get uninstalled.
Quick Uninstall
This is not recommended because it will remove all packages installed along with XFCE. This may include packages you previously installed (like Thunar).
Note: this uninstall command assumes XFCE was installed on a fresh Ubuntu 11.10.
1) Open a terminal (instructions can be found here).
2) Enter the following command (it is very long and a single line):
sudo apt-get remove --auto-remove desktop-base exo-utils gtk2-engines-xfce libexo-1-0 libexo-common libexo-helpers libgarcon-1-0 libgarcon-common libglade2-0 libjpeg-progs libkeybinder0 libthunarx-2-0 libtumbler-1-0 liburi-perl libxfce4ui-1-0 libxfce4util-bin libxfce4util-common libxfce4util4 libxfconf-0-2 orage tango-icon-theme thunar thunar-data thunar-volman tumbler tumbler-common xfce-keyboard-shortcuts xfce4 xfce4-appfinder xfce4-mixer xfce4-panel xfce4-session xfce4-settings xfce4-utils xfce4-volumed xfconf xfdesktop4 xfdesktop4-data xfwm4 xfwm4-themes xscreensaver xscreensaver-data
It is easiest to copy the line and paste it into the terminal (right-click on the terminal and select Paste from the popup menu).
Preferred Uninstall
The better way to uninstall it is to follow the directions on this page. This will allow you to uninstall only the packages that were installed when XFCE was installed. The instructions are generic and work for any installed package.
This involves (1) identifying which packages were installed and (2) creating a command to uninstall only those packages.
This leaves intact any packages you may have installed (like Thunar) instead of blindly assuming that they were installed along with XFCE.