Yes, they're hot swappable. This is achieved mostly by the physical configuration of the socket pins. A few pins stick out or are recessed further than the other pins in the socket.
When you insert a PCMCIA or CF card into an ADS system, the ground pins connect first, followed by the data and power pins. The shortest pins on the socket are the card detect pins, which tell the driver to energize the power and data lines.
Of course, any hardware you hook up to the CF bus becomes more like a bus card than a PCMCIA/CF card. In such "hardwired" configurations, you have to take the normal precautions of turning off power to the system before connecting hardware. If you use a PCMCIA/CF header to connect your hardware, or use your own software drivers to de-energize the CF bus when hardware is disconnected, you can implement your own hot-swap card system.