There is no available GUI interface that allows you to completely remove a package and all the other packages installed with it in one step.
However, you can use Ubuntu Software Center to completely remove all packages – you just have to do it one package at a time.
(1) determine which packages were installed – this means looking at the history.log to find out which packages were installed.
(2) enter the package names (one at a time) into the Ubuntu Software Center and click on Remove
There is an updated (for Ubuntu 12.04)) version of this article here. It is shorter and better expains the problem and solution.
So far, I have not found any software package (with or without at GUI) that completely uninstall what was installed – they all use sudo apt-get remove --auto-remove <package name> (or its equivalent). As my article explains, in many cases, this leaves behind unneeded and unused packages.
There are a number of packages that claim to clean up your system, but what they do is run sudo apt-get autoremove which removes some left behind packages. Others also run sudo apt-get purge which cleans out the repository cache (these are the files that are downloaded when you install something), but it does not remove the installed packages.
This is a serious problem and people should complain about it to Linux developers.

You can definitely do sudo apt-get autoremove to clean up unused dependencies (which I mention in the article) – and this works fine, but it does not uninstall everything, neither does <>deborphan.
For example, if you install the Xubuntu desktop, in addition to installing the desktop, it also installs Abiword (among other things). Even if you use the autoremove option of apt-get or you run deborphan, Abiword and all the the applications installed along with Xubuntu desktop do not get uninstalled. Since none of those applications are orphaned (because you can always install them yourself without installing the Xubuntu desktop), the only way to clean them up is to remove them one by one.
The proper way for the uninstaller to work is to read the installation log to determine what needs to be uninstalled.
My solution (page 3 of this article) shows the user how to do this manually.
An even better uninstaller would tell you which applications (if extra applications had been installed) were being removed and give you the choice of keeping the applications you want. On the other hand, a good installer would tell you which extra applications are being installed (apt-get at the commandline does this, but the info is not displayed in the Ubuntu Software Center) and give you the choice of installing them or not.
Thanks for your comment.

I haven’t run VirtualBox inside of Ubuntu (I use it to run Ubuntu inside of Windows), but … the command to remove it should be sudo apt-get remove –auto-remove virtualbox-qt.
Entering sudo apt-get autoremove instructs the installer system to automatically remove unused packages from the computer.
Hope this helps and thanks for your question.