It is generally not advised to drink the local water although brushing your teeth in it should not be a problem. Bottled water is available everywhere and is not expensive. (Please dispose of your bottles/refuse responsibly as there is a big environmental challenge with litter and refuse!)
Most hotels will provide a bottle or two of complimentary water (in addition to whatever is in the mini bar in your room). It is suggested that you pop these complimentary bottles into the mini-bar fridge so they are cold for when you need them.