Moving from Non-Banking/Finance to IB

Those who have worked earlier in a non-finance field like a physics/philosophy grad or even a science grad, please share your experiences on moving into Investment banking or a related non-banking role like HF or PE. Please also share why and how you managed to make the switch.