Just that expected value isn't the only factor to consider. But I was also thinking about it with respect to the Chargers game on Sunday and in general, not specific to either that game or the BYU game

It also applies in some situations when you're deep in your own territory and losing. Imagine you're down by 13 towards the end of the 3rd and have 4th and 2 from your own 24. In that scenario the value of getting to your own 26 isn't huge as you still have to drive, but giving the opponent the ball that close to your own end zone (and already in FG range) carries massive downside as it makes it a 3 possession game.