Thanks, that's definitely a silly issue. I would recommend binding the Zoom function under Settings -> Controls -> VR which toggles you between a moderate zoom and a 1-to-1 VR FOV.
Just to clarify, For some of the planes the 2D zoom and the VR zoom are in the same direction, others not. I didn't note down which ones are which, but can do if it's helpful.