These instructions are for Ubuntu 11.10 on which LXDE was installed using either the Ubuntu Software Center or sudo apt-get on the command line.
The instructions may be the same for other versions, but no guarantee is made.
When the LXDE interface is installed in Ubuntu 11.10, it installs several additional packages and libraries. Several packages (like arj and gpicview) do not get uninstalled when LXDE is uninstalled.
Quick Uninstall
This is not recommended because it will remove all packages that are normally installed with LXDE. This may include packages you previously installed (like arj or gpicviewer).
NOTE: this uninstall command assumes LXDE was installed on a fresh Ubuntu 11.10.
1) Open a terminal (instructions can be found here).
2) Enter the following command (it is very long and a single line):
sudo apt-get remove --auto-remove arj galculator giblib1 gpicview leafpad libfm-data libfm-gtk-data libfm-gtk1 libfm1 libglade2-0 libimlib2 libjpeg-progs libmenu-cache1 libobrender27 libobt0 librpm2 librpmbuild2 librpmio2 librpmsign0 libxmmsclient-glib1 libxmmsclient6 lxappearance lxde lxde-common lxde-core lxde-icon-theme lxinput lxmenu-data lxmusic lxpanel lxrandr lxsession lxsession-edit lxshortcut lxterminal menu menu-xdg obconf openbox openbox-themes p7zip-full pcmanfm rpm rpm-common rpm2cpio scrot xarchiver xmms2-core xmms2-plugin-alsa xmms2-plugin-id3v2 xmms2-plugin-mad xmms2-plugin-vorbis xscreensaver xscreensaver-data
It is easiest to copy the line and paste it into the terminal (right-click on the terminal and select Paste from the popup menu).
Preferred Uninstall
The better way to uninstall it is to follow the directions on this page. This will allow you to uninstall only the packages that were installed when LXDE was installed. The instructions are generic and work for any installed package.
This involves (1) identifying which packages were installed and (2) creating a command to uninstall only those packages.
This leaves intact any packages you may have installed (like arj or gpicview) instead of blindly assuming they were installed with LXDE.