workplace ambush

Web dev dude appears in my cube. We had talked at a drinks thing over a month ago, where I mentioned my boyfriend, staying over at my boyfriend's house, taking BART to work from my boyfriend's house, and my boyfriend at least a dozen times.
"Wow, I didn't know you ever made it to this side of the office."
"I came over here just to say hi to you."
Awkward.
S walks by and I engage him in accounting talk for a few minutes while he notices ominously hovering web dev dude and comes up with an excuse to vanish.
"You guys seem pretty busy."
"Yeah, I'm in a job transition."
Blah, blah, we talk about my internal transfer. Not sure why he would be coming over to my cube during the work day to talk to me a month after the last time we had an actual conversation, when he knows I have a boyfriend.
"So, what are you doing for lunch today?"
I go into some excuse based loosely on true facts about how I'm supposed to call some friend of a friend during my lunch hour.
"Okay, see you later," he says while walking away.
It feels rude that I blew him off and didn't suggest we do another time, because he might just be trying to be friends, and I would have made an effort with someone else, but I'm pretty sure I saw this scene in our sexual harassment training video, where module after module this woman leads the IT temp on until he's totally obsessed with her.
Not that that's realistic, but it would have been easier to say yes to a lunch invite if he was bringing friends, or if it was otherwise disguised it as something not totally inappropriate like trying to date your coworker who has a boyfriend.