Doable. Lots of stigma because is really hard to justify switch from one to the other for someone's own volition. Hours are slightly better but not life changing and travel can nullify that benefit, in US compensation - if successful in both - is 30-50% of what someone would make in CIB (mind you, not 30-50% less), lots of other attractive exit opportunities (e.g. Corporate finance in industry, PE, etc).
that said, recruiting will love someone from IB because it becomes a poster child!
Andrea