I personally have no opinion about whether it's "immoral" for someone to renege on a football scheduling agreement. But you seemed to have make a statement (whether intentional or not) that behavior isn't immoral if there is a contract in place and the parties aren't violating it.
Like I said, no personal opinion here. Just an interesting discussion IMO. In both cases you have a contract to do certain things (play games on your opponent's field, stay faithful in marriage). In both cases you have penalties set for anyone that violates the agreement ($$ buyout, alimony/child support/asset division). From a morality standpoint (meaning what is right or honorable – not something related to sexual sin, which often is the definition in Mormondom), I think someone has a point if they ask why one behavior would be immoral but the other isn't since both were governed by agreements with penalties set to punish someone violating the terms.