Although the color-up process CAN help a lazy floorperson track a player, that really is NOT the reason. The dealers ARE telling you the truth. When their rack gets low on chips, they have to order more frequent fills, which costs the casino time & therefore money. I've seen many a dealer get scolded for letting someone get away without coloring up, even though it often is not their fault.
When I want to bolt without a color-up, I usually just put on my "hard of hearing" act & walk away acting like I didn't hear anything at all. After hundreds or perhaps thousands of times doing this, I've yet to have someone actually chase after me.