Will an HDMI splitter take the HDMI inputs from both devices and enable them to both work on the TV? I'm new to firestick (and I have a dumb but excellent TV).

You need an HDMI switch, not a splitter. A splitter allows one source to feed multiple tvs or monitors. A switch allows a single input on a TV to be switched between multiple sources, which is what you want to do.

We have used an amazon fire stick many times as well as an Apple TV. Just plug in your fire stick to your HDMI outlet on your tv and, presto, your done. Works as normal, as if you were in the states. Problem solved!