Search engine is a system that sorts and classifies Internet information resources and stores them in network database for users to query, including information collection, information classification and user query.

principle of operation

1. Crawl: Search engines follow the links of web pages through specific software, and crawl from one link to another, just like spiders crawling on cobwebs, so they are called "spiders" or "robots". Search engine spiders have certain rules to enter, and they need to follow some commands or the contents of files.

2. Capture storage: Search engines capture web pages through spider tracking links and store the captured data in the original page database. The page data is exactly the same as the HTML obtained by the user's browser. Search engine spiders also do some duplicate content detection when crawling pages. Once they encounter a lot of plagiarized, included or plagiarized content on websites with low weight, they are likely to stop crawling.

3. Preprocessing: Search engines will preprocess the pages crawled by spiders.