I’ve read this in a few different places with a few different explanations.

Readers at Thrifty Fun explain:

“When they lick off the butter, they remove traces of the old territory (their comfort zone or 'home') which makes them less likely to try to stray to re-find their comfort zone."

Susan McGarvey from the Frugal Duchess:

“Next time you move, butter your cat's paws before you let it outside for the first time at your new residence. Instead of darting out the door in a panic, the cat will sit down to lick the butter, which gives it a little time to become aware of its surroundings. We've used this trick in our family for years, and we've yet to lose a cat!”

Cat World calls this a myth but it seems a lot of people stand by this idea.