Boyfriend upset by ex-boyfriend?

My current boyfriend of 9 months dislikes my ex-boyfriend. my ex and I only dated for 6 months, but remained very close and still had feels for much longer. Now, two and a half years later I've moved on, but want to stay friends. My ex still had feelings for me for a while, which made my bf uncomfortable so I distanced myself from my ex for a while. Now my ex and I are only casual friends. my bf doesn't want me to go anyplace like the mall or out for coffee with my ex because its too much like a date and he doesn't like me spending time with him at all if there aren't others there. I feel like i can't talk about my ex with my bf which upsets me. I'm not sure if i'm wrong, but i think its ok for me to spend time with my ex and i feel that out in a public place would seem better. I want a friendship with my ex, i don't want to cut him out of my life, and i can't always make sure someone else is there. but i don't want to upset my bf either. am i completly unjustified in my thinking?