I don't believe you need do a "remove device" on all non storage related peripherals. e.g. keyboard, mouse, printer. They don't have any storage capacity for any data. You just unplug your USB keyboard, mouse, printer. It's not gonna hurt your keyboard/mouse/printer a bit.
It's the data...