Do wine label dimensions change for different bottle sizes like magnums?

Yes, label size scales with the bottle. Larger formats need proportionally larger labels, and smaller bottles need smaller ones, so scaling straight from a 750ml size can leave you off.

The safest approach is to measure the actual bottle you are labeling rather than assuming a fixed ratio, since shoulder shape and curvature vary by producer.