AITA for telling my coworker their “special diet” is just an excuse to avoid contributing at potlucks?

I work in a small office where we frequently organize potlucks for team-building events. Everyone usually brings a dish to share, and it’s something we all enjoy—except for one coworker, Jake (34M).

Jake claims to follow a “special diet” for health reasons, but he’s never been specific about what it is. Every time we have a potluck, he doesn’t bring anything, yet he always eats everyone else’s food. He’ll fill his plate and say something vague like, “This fits my diet,” without explaining further.

Last week, we had another potluck, and the pattern repeated. I was annoyed because I brought homemade food, and Jake took a large portion without contributing anything. I finally said, “Jake, if you can eat all this food, your ‘special diet’ doesn’t seem like much of a restriction. It just seems like an excuse not to bring anything.”

The room got awkward, and Jake looked embarrassed. He muttered something about “not having time to cook” and left early. A couple of coworkers later told me I was too harsh, but others quietly agreed that Jake has been taking advantage of the potlucks.

Now I feel conflicted. I didn’t mean to humiliate him, but it’s frustrating to see someone freeload off everyone else’s efforts. AITA for calling him out?