IMO, the problem is the clause uses the word 'deteriorating' in regards to his father's health. And that's where the dispute is. Anthony would have said that he's worried about his father, and the Raiders said "tell you what, if things go bad, if he gets any worse, you're free to go". The Raiders are claiming he's gotten better, so the clause conditions have not been met.
FWIW this is the only leg the Raiders have to stand on, in that the clause is only activated if he gets worse. Legally they seem to be in the right. But it will be disputed in that the clause was there to protect Milford's interest in his family, and that even though his father hasn't gotten any worse, at least on the surface, as we all know with heart issues, it's like being on a tightrope and the next attack could be a second away. For this reason, the clause doesn't protect Milford, and arguably only allows Milford to be release potentially once it's too late, and therefore that will be the basis for the fight.
Or of course the Raiders should accept that it's a fight that regardless of the outcome they'll look bad, do what's best for the kid, and just drop the bone. I'd be incredibly surprised, stumped even, if Stuart's ego allows this.