Games where I have perfect knowledge of the state of play, and where one player moves first, I don’t enjoy much. For each of these games, there provably exists a strategy where the first player that moves can only win or draw

That doesn’t seem quite correct for any game meeting those criteria (I’d also add that the game is deterministic - no true randomness in the game either, since that is distinct from state - otherwise the outcome could trivially depend on random events). There are two other possibilities for a deterministic game: that optimal gameplay by both players will always end in the second (or another player if more than two) winning, or that optimal gameplay by both players will result in a game that never ends (impossible for games with a finite number of states, and rule that the game ends in an outcome if the same state recurs too many times - like chess).

A trivial example of a (poor) game that would meets your criterion but where the first player loses under optimal strategy: Players take turns placing a counter anywhere in the play area from an infinite supply of counters. Players cannot skip a turn. If there are an even number of counters on the board after a player’s turn, the player who placed the counter can optionally declare victory and win. Not a game I’d play, but it does prove there exist deterministic open state games where one player moves first where the first player will not win or tie.

In a 3+ player deterministic open state game, the actions of a player who goes on to lose could impact which of the remaining players win (they are essentially just a different source of non-determinism).

I think it is correct to say that any two-player deterministic open-state game which can only end in a draw, win, or tie, for any fixed initial conditions, there exists a strategy for one of the two players that will ensure that one of the three outcomes occurs: the game continues forever, that player draws, or that player wins. That can be proved by contradiction: either one or more move in the strategy decision tree can be improved to make the player win, which contradicts the strategy not existing, or the other player can rely on the strategy not existing for the first player to devise a strategy, which also contradicts no strategy existing for either player.