When we switch from a single config to split configs, make sure the old
config is removed. Leaving it in place prevents logstash from starting,
since we have multiple inputs attempting to listen on the same port.
Change-Id: I12e9da295e892ec70bcd9717fbdc0526b0822b86