Ex-adviser: Pence was upset about backlash over coronavirus op-ed