JWDigitalResearch, and why it visited your website

It visits the websites of a sample of businesses that appear in Google's results for their trade in a few towns across the UK. On each website it reads the home page, up to seven pages from the menu and the sitemap. Some pages it opens in a browser, so it sees what a visitor sees.

It notes what kinds of thing the website has: for example a prices page, a contact form and the types of its fields, a booking button, a map or a link to social media. It counts them across many websites so we can see what is usual for each kind of business.

To tell what each part of a page is, an AI model reads the page's words and pictures of the page as a visitor sees it. The model only says what kind of thing each part is, such as reviews, prices or a contact form. Nothing it reads is kept for longer than the copies described below.

• Only kinds and counts. We never keep your words, your photographs, your design, or any name, number or address from your website, and we never publish anything from it.
• Copies for checking, for 30 days at most. The pages as they were read, and the pictures of them, are kept outside our products while we check the work, then deleted.
• Nothing more than reading. It does not fill in forms, sign in, accept or decline cookie banners, or follow adverts, and it never tries to get past anything that blocks it.

• It names itself as JWDigitalResearch in every request, with a link to this page.
• It reads your robots.txt first and follows it, including any Crawl-delay.
• It waits at least three seconds between pages on your website, and reads one page at a time.
• If your website answers that it is busy or refuses it, it stops and does not come back in that study.

Add these two lines to your robots.txt and it will not read any page of your website:

User-agent: JWDigitalResearch
Disallow: /

Or email enquiries@jw-digital.co.uk with your website's address. We will leave it out of our research and delete anything we still hold from it.