I've the same problem in windows 10 with 765M Nvidia card. I fix the problem change the resolution in advanced settings of screen in windows but the problem continues when i try to open advanced mode in resolume for working with masks, slices.... I updated the nvidia card driver, actually i've 358.5...