I’m no medical professional, but if I recall correctly, lactic acid is transported by the blood stream, especially when muscles aren’t built up to deal with the excess acid locally. Exercise will alter your blood’s pH even if you’re in shape. Blood also turns more acidic simply by the presence of more CO2 in the system, but that shouldn’t affect the pH too much.

That said, I can find no source online indicating that the change in pH is the cause of the burning feeling when exercising while out of shape.