Crawler Configuration
The Site Search Crawler can use web technologies to improve the granularity and scope of a crawl.
But there are things that the Crawler cannot understand, like IFrames or client side JavaScript...
Use the sidebar or explore the links below:
- Advanced settings
 - Canonical Crawling
 - Constant Crawl
 - Content Inclusion & Exclusion
 - Crawling Password Protected or Private Content
 - IFrames
 - Image Crawling
 - Language Optimization
 - Meta Tags
 - PDF Crawling
 - robots.txt
 - rel="nofollow"
 - RSS and Atom
 - Sitemap XML
 - URL Inspector
 - Video Crawling
 
Stuck? Looking for help? Contact support or check out the Site Search community forum!