There are bus companies that run regularly, that will cost you $9 on Giant Ibis, who are one of the most reputable companies in Cambodia.
You can rent a private taxi which will cost around $40 – this is worth it if you are a group of four or if you need to hot foot it down. This is the option we took as we wanted to leave super early to have more in the day there.
The third option, a more scenic route, is by taking the train from Phnom Penh - the train takes about seven hours, with two thirty minute stops, at only $7 per person it is a bargain! The only down side of the train is that it only runs on the weekend so you’d have to plan accordingly. What some friends have done recently is they took the train down, for the experience and then a taxi back to Phnom Penh during the week.
We stayed at a hotel called Natural Gardens, it being my birthday spoils weekend we had a lovely room right on the river. It was very nice however I do think you can find other places at a similar price with a bit better value for money. There are few hostels, hotels and resorts on the other side of the river that are propped up on the water with stilts. The best way to get around Kampot is to hire a scooter or, if you are feeling brave, a dirt bike. We went for a dirt bike which was great fun and good on the dirt roads however you do need to know how to handle it. A scooter can get you where you need to go, just a little slower. There are loads of places that rent them out, they charge around $5 a day. Try find a place that has the number on the key and will come rescue you if you break down!