I mean, it wouldn’t have a choice. It’s a gigantic, elaborate mathematical equation. It can’t choose it’s responses any more than 2+2=? can.

If all the material it was trained on says “4”, it has no choice but to say “4”.