Can the Google-Bot fill out and crawl forms - SISTRIX Login Free trialSISTRIX BlogFree ToolsAsk SISTRIXTutorialsWorkshopsAcademy Home / Ask SISTRIX / Crawling and indexing / Can the Google-Bot fill out and crawl forms

Can the Google-Bot fill out and crawl forms

From: SISTRIX Team 05.02.2021 Google-Index, Google-Bot and the Crawling Process What is the Google Everflux? Robots meta tag vs. robots.txt: what are the main differences? What is an HTTP referrer? Our web site is no longer in the index - have we lost our rankings? What is a User-Agent? What is Google Search Console and How To Get Started Web Crawlers: How do They Work? Changing Google Search through Entities What is the X-Robots-Tag? What is the Mobile First Index? Rich Snippets: What are the advantages? Can the Google-Bot fill out and crawl forms Crawl Budget: What does this mean? These are the CTR's For Various Types of Google Search Result Crawling and Indexing for extensive websites Google SERP Features: Result Types in the Search Results Why does the amount of indexed pages fluctuate so much? How can I quickly get a new page into Google's index? Why does a blocked, noindex URL show up in the search results? Is a website with and without the www harmful? Shelf space optimisation on Google Find out how many pages of a domain are indexed by Google The consequences of negative user-signals on Google's rankings Why am I getting different values for indexed pages in the Google search, the GSC and SISTRIX? How can I remove a URL on my website from the Google Index? Back to overviewThe Google-Bot will generally try to fill out and post forms on a page, in order to discover new content and URLs that are not directly viewable otherwise.ContentsContentsHow can I make the content behind my forms accessible to Google Video explanation by Matt Cutts Google on this topicHow is Google finding pages which don't have any links to them?Additional information by Google on this topic Google will decide on an individual basis if a FORM-Element on a page is considered to be useful and then try to fill out that form using a small number of different natural requests, made to simulate an actual user. Google only crawls forms which use the GET-method and do not ask for personal information. Additionally, the form should be made up of no more than two input fields.

How can I make the content behind my forms accessible to Google

For this, you would have to set up your forms in a way that the Google-Bot can fill them in and crawl their content. You cannot, however, force the Google-Bot to crawl the content. A relevant text near the FORM-Element, describing the reason behind the form, can also be helpful. The use of the so called AutoCompleteType-Attribute does not help the Google-Bot fill out the form, but is helpful for users. To make sure that you do not rely entirely on having your forms crawled, you would have to link all the pages behind your form with each other and have at least one of these pages accessible through your domain’s startpage. You should also make a HTML- or XML-Sitemap available for all the pages that can be reached through the form, so that Google has an easier time finding them all.

Video explanation by Matt Cutts Google on this topic

How is Google finding pages which don’ t have any links to them

How can Googlebot crawl and index pages that don’t have any links to them on my website? I find each day two or three pages in the index that don’t have any links to them on my site. The pages are generated by the search field of my website.

Additional information by Google on this topic

Crawling through HTML formsGET, POST, and safely surfacing more of the web From: SISTRIX Team 05.02.2021 Google-Index, Google-Bot and the Crawling Process What is the Google Everflux? Robots meta tag vs. robots.txt: what are the main differences? What is an HTTP referrer? Our web site is no longer in the index - have we lost our rankings? What is a User-Agent? What is Google Search Console and How To Get Started Web Crawlers: How do They Work? Changing Google Search through Entities What is the X-Robots-Tag? What is the Mobile First Index? Rich Snippets: What are the advantages? Can the Google-Bot fill out and crawl forms Crawl Budget: What does this mean? These are the CTR's For Various Types of Google Search Result Crawling and Indexing for extensive websites Google SERP Features: Result Types in the Search Results Why does the amount of indexed pages fluctuate so much? How can I quickly get a new page into Google's index? Why does a blocked, noindex URL show up in the search results? Is a website with and without the www harmful? Shelf space optimisation on Google Find out how many pages of a domain are indexed by Google The consequences of negative user-signals on Google's rankings Why am I getting different values for indexed pages in the Google search, the GSC and SISTRIX? How can I remove a URL on my website from the Google Index? Back to overview German English Spanish Italian French