To be pedantic, Remote Operated underwater Vehicle (ROV) does not inherently disqualify an unmanned remotely operated submarine, albeit based on the use case one may chose a certain method of locomotion over another. If you have a rough layout for a ROV of your description that is within the price constraint, honestly I'd love to hear it.
Perhaps brush-less DC motors would be better suited to a wet environment, with enameling applied to all solder joints, they should be able to be entirely submerged, allowing more flexibility in regards to the hull design. With appropriate ESC's we'd be talking at least $20-30 for 4 motors and ESC's, unless you made your own. I would be concerned with brushed dc motors shorting unless steps were taken to separate the motor from the water source.
For buoyancy control, I would still recommend lead weights, as to mitigate the risk of shorts by allowing a roughly self contained air filled device to neutrally buoyant. For 2 liters you's need about 2 kilo. If you can find cheaper weights, like gravel etc and affix them to the bottom they should work similarly. Use something cheap, and dense. 10 bucks should take care of that.
That would leave from 10-30 bucks to create a frame to support the motors outside the main hull, acquire necessary control circuitry, Power supply, and other extraneous expenses. The advantage of the ESC route would be that the ESC's should be controllable from the Arduino without a shield.
The submarine type approach should be better suited towards the hardware already possessed, and probably about 10-20 bucks cheaper