There is no software-defined limit on the number of endpoints and tunnels you can have; you'll be limited only by your license and the amount of memory available to VNS3.
Regarding memory constraints, each endpoint consumes ~20MB of memory, and tunnels consume ~10MB.
If your current running setup has 500MB of free memory, you could define 1 endpoint and 48 tunnels, 2 endpoints with 23 tunnels each, or 16 endpoints with 1 tunnel each and have 20MB left over.
These numbers are based on a worst-case scenario, and should be interpreted and used only as a baseline; real-world tests generally perform much better.
Keep in mind that these numbers are not related to IPSec throughput, which depends on your instance size - specifically, available CPU cycles (primarily for encryption) and your host / hypervisor's network capabilities.