Your video resolution seems to be the problem.
You are outputing from Serato as 400x225 pixels. Your projector has a 1200x800 resolution. If you are displaying your image in full screen, you are incresing its size about three times and also distorting it.
Another very important thing that also took me a while to fully understand: Arena composition sizes have no relation to the projector output resolution.
Why are you scaling the output down in Serato? Your video resolution is 640 x 360 (16:9) but you are outputting 400 x 225 (16:9). Then you are scaling up in Resolume to 1280x800 which is a different format (16:10) so essentially you are scaling up 320% horizontally and 356% vertically which is not going to look clean since you're stretching the image vertically.
Why don't you output 640 x 360 out of Serato (or even better find the same video in 1280x720), set your composition to 1280 x 720, and your projector to 1280 x 720 as well.
If 1280 x 720 is not available on your projector, use the advanced output to create a 1280 x 720 slice within you 1280 x 800 projector and you'll be left with a black band of 80 pixels which you can set top, bottom or split however you want.