Think you might have to change in Swansea. Long time since me caught the London to swansea train. Trains in UK are very expensive too. Might be cheaper to rent a car. Plus a car would save the hassle of changing trains, waiting, and the possibility of cancellations. Cancellations are rather frequent. Good luck and enjoy your trip.

If you can make a short detour and visit Gower. 20 odd miles outside Swansea and, if me remembers correctly, the first area of outstanding natural beauty in the UK.