Keep talking and narrating, a lot. It does make a huge difference. Particularly if you do it in Swedish. Your boy will learn English naturally from everyone around him, but he won't learn Swedish unless you use it a lot and encourage it.

Also, kids understand a lot more than it appears. My daughter learned yes and no fairly early. Not just the souds, but the meanings. I could ask questions like "do you want chicken?" and she was able to say yes or no. She would've never been able to say "I want chicken" at the time. We also taught her some sign language and that was very helpful when she was still learning how to talk. She learned the sign for water way before she could say it. So she could ask for it even though she couldn't express it out loud.