Gift gone bad?

I have this coworker. I think we both like each other, no one has asked anyone out but we’ve seen each other outside work a couple times and I’d said as far as coworkers go, we’re close. A couple months ago he got a store gift card for his birthday from our company. He told me he didn’t go there and he’d never use it. I told him you can buy anything at that store and I’m there all the time. We never talked about it after that. Two weeks ago he came in work and asked me if I wanted the gift card because he wasn’t going to use it. So I took it and a couple days later I went to the store and got him candy he likes and a hat. He told me he didn’t like getting gifts so I didn’t wrap it and just put the bag on his desk. He said thank you and asked if I got it with the gift card. The bag sat on his desk for four days before he opened the candy (insane, because he never passes up candy or food, ever) and it’s been over a week since he took the bag home and he hasn’t worn the hat. Did I freak him out? I mean I bought it with HIS gift card. I just thought it was the polite thing to do since he gave me a gift card and I still had over half the gift card left. Or, am I completely overthinking this.