Steer spiders to crawl your site

For regular and proper spidering of your sites do the following:

Provide unique and relevant website content. Add lot of content in relational pages as well. Add content on a regular basis on blogs, in case of websites create at least one relational page a year.

Get back links or in coming links from related sites and pages hosted on various servers. Acquire inbound links for the relational pages.

Create external links from your site to authoritative topic relevant sites for additional information. Limit such links.

Web design play a role in seo.Create good site architecture which is well themed, link relational and utility pages using text links. Inter connect your pages using text links.

Create both site maps XML one for search engines and HTML site map for users.

Use search engine friendly, short but keyword rich URLs.
Log each crawler visit and keep these data forever.
Use logs to improve your internal linking.

Use robot exclusion protocol to keep spiders away from pages that are not apt for caching. Such that the robots META tag to ensures that only one version of each page on your server gets indexed.

Use rel="no follow" in your links, when you cannot vote for the linked page

Search engines consider pages with no inbound links useless and usually do not bother to index indexing them.

Ranking algorithms analyze a site's internal linking and user friendly navigation. Deeply buried pages should be easily found within one or two clicks.