You really have a large amount of them.I have lots of the passion flower cats but knowhere near the amount you have.
They look quite similar to catalpa worms. They eat all the leaves off our trees but do no harm to the tree itself.