The joke is a racial stereotype. The black guy asks for his lunch, and he says no... but when the white guys asks, He says yes.
lol what? I got the pun with the first one, but when the second guy asks... the joke kinda makes way for mystery. Why?! What did that other man do or say to deserve that lunch? Was the wordplay but a little ploy in a much bigger scheme of things to be?! Feels like ending after the first encounter would've been just right IMO. Gets the joke across great, if that is the main joke, then. I love quick wordplay puns like that.
you can have my lunch