A public site is a site that can be viewed by anyone, functioning like most other websites on the internet. This means that if I share the link with a person and s/he shares the link with the third person that person will be able to access my site.
It also means that, unless you have turned on the Advanced Settings for your site's link, to capture your visitors' names and email addresses, your site's visitors will appear as Anonymous.
Typically, the default Visitor Access setting for a site is public. For more information on how to change viewing permissions, read this article.