Originally Posted by TheJoe
I realise this reply is a little late but it's totally fine for you to do that. ubuntu-desktop is a meta package for GNOME. It won't remove anything if you uninstall it.
You can reinstall it again afterwards if you like.
Thanks for the info!
I gave up compiling it 3~4 weeks ago.
I hope it continues the compiling successfully after I've removed the package.