XUL is like HTML for native GUIs.
XPCOM is like COM. It lets you get pointers to components inside Mozilla, and it lets you talk about their interfaces as well.
XPConnect is what lets JavaScript? talk with Mozilla guts, the XPCOM system. You can imagine it like a tunnel between Javascript code, and Mozilla guts.
xpcshell is a command line interface to Mozilla JavaScript?. You use it like "python" on the command line- it gives you a shell that talks straight into Mozilla, via JavaScript?'s XPConnect bridge into it.