What we don't often think about is this is really true only at the equator, and even there it's only entirely true during the solstices. For people anywhere else, or at any other time, the sun actually rises in the northeast and sets in the northwest (if you're in the southern hemisphere) or rises in the southeast and sets in the southwest (if you're in the southern hemisphere). Or at least it would, if the earth weren't tilted on its axis.
Since the earth is tilted, not only does the sun generally not rise and set at locations 180 degrees apart from each other, the location of sunrise and sunset wobbles as the year goes on.
When you're north of the Arctic Circle, things get really weird.
At the summer solstice, the sun doesn't set at all, and during the winter solstice, it never rises. The rest of the time, it makes circles in the sky. The circles wobble as the year goes by...during the summer, most of the circle is above the horizon, and as winter comes, the circle sinks below the horizon. (So, if you plot the path of the sun in the sky--when it is in the sky--over the course of time, it actually does a spiral.)
Last night, Eve and I climbed to the top of Anvil Mountain just outside Nome, Alaska (which is near enough to the Arctic Circle to see some of the weirdness) at 2 o'clock in the morning to watch the "sunset." I say "sunset" because it's still pretty much full daylight out. The sun dips just barely below the edge of the horizon, but it doesn't stay there, and it comes up again shortly thereafter...meaning we saw a simultaneous sunset and sunrise.
The red in the sky on the left of this panorama is the sunset. The red in the sky on the right is sunrise. The sun is traveling in a shallow arc that just barely dips beneath the horizon.
Click on the picture to embiggen!