I believe this is a bug on Safari, but I figured out a way to work around it. It has to do with certain elements and their styles. By process of elimination, I narrowed it down to a few "offending" HTML elements. Deleting width: 100%; from some elements and CSS styles, as well as other static widths like width: 120px; would start to get reduce problem. I say "start to reduce", because the margin on the right became smaller, but didn't go away completely. I then started playing with other CSS attributes like margin and padding. After getting rid of some left and right padding from some elements, the problem finally went away. But this wasn't really acceptable, as those styles were there for a reason.
The solution was to wrap everything in a container element, size that appropriately, and set overflow: hidden; in CSS. Setting overflow: hidden; to the body or html tags would work too, but that did funky things with vertical scrolling in Mobile Safari. In my case, there was already such a container element, so all I had to do was add the overflow property to it.
Like I said, I think this is a bug in Safari. When you rotate from Landscape to Portrait, everything should be resized back to fit portrait mode. Visually, everything does look like it was resized properly. However, Safari must have thought something wasn't resized properly, so it displayed the page wider than it really was. This works just fine in Chrome on an Android device. I also added different background colors and borders to highlight which element might be causing the page to stretch beyond the width of the device screen. Visually, there was no apparent culprit.
If you're thinking it might be a width: 100% plus padding issue, I had the same thought. But then deleting either the width or the margin/padding alone should have fixed the issue, which it did not. Not a single element was sitting beyond the edge of the screen. There was nothing but empty space there.
I also removed height=device-height from the viewport meta tag, as this didn't play nicely with Mobile Safari.