you might. It's better to be safe than sorry. If you don't have the towel completely around them (accidentally, or even on purpose) then it's best to wear them. I would just to be sure.
It would be a good idea as the tile chips could cut through the towel and come flying out of nowhere unexpectedly.How to make a mosaic tile bathtub?