LWF SEO Web Spider: Troubleshooting

The App Does Not Open

1. Confirm that the download matches your operating system and processor where separate builds are supplied.
2. Move or install the app in the normal Applications or Program Files location.
3. Confirm that the download came from London Web Factory.
4. Check whether security software quarantined part of the application.
5. Download a fresh copy if the archive or installer may be incomplete.

On first launch, the licence window must be accepted before the main application opens. Declining closes the app.

A Website Does Not Load in the Browser

• Confirm the address is an HTTP or HTTPS URL.
• Try reloading the page.
• Check the same URL in another browser on the same computer.
• Check for a DNS, proxy, VPN, firewall or TLS error.
• The website may block the app's browser User Agent or network address.
• A page can require authentication, cookies, consent or a geographic location that is not available in the current session.

Text that does not look like a website address is sent to Google as a search query.

Analyse Page Is Empty or Incomplete

Page Analysis reads the rendered DOM at one moment after loading. It does not automatically scroll, click consent controls, expand accordions or wait indefinitely for delayed content.

Try:

1. waiting for the visible page to finish loading;
2. completing any required consent or login in the Browser tab;
3. selecting Analyse page again;
4. checking whether the missing content is inside an iframe or loaded only after interaction.

The Browser tab and crawl window use different sessions. Content visible because of browser cookies might not appear during a crawl.

A Crawl Does Not Start

• Only one crawl can run at a time.
• Wait for an active external header check to finish.
• Enter or open a valid HTTP or HTTPS address.
• Stop an existing crawl before starting another.
• Standard users can select Gentle speed only.

Starting another crawl replaces the current working crawl database. Export important results first.

The Crawl Finds Very Few Pages

Check the following:

• The crawler remains on the exact starting host. www.example.com, example.com and shop.example.com are separate scopes.
• Navigation might be generated only after clicks or scrolling.
• Links might use forms, script handlers or unsupported schemes rather than normal <a href> elements.
• A consent page, login page or anti-bot response may be what the crawl session receives.
• Important URLs may be reachable only through a sitemap; the current queue follows discovered same-host page and resource URLs.
• Some state-changing or sensitive URL patterns are deliberately excluded.
• Retained query parameters may create more identities than expected, while fragments and tracking parameters are deduplicated.

Open the first page in Crawl Results and inspect its Outlinks and Resources tabs to see what the crawler captured.

The Crawl Finds More URLs Than Expected

Functional query parameters are retained. Faceted navigation, calendars, internal searches and generated parameter combinations can therefore produce many distinct URLs.

Check the Address column for recurring parameter patterns. Pause or stop the crawl if the queue grows beyond the intended scope.

A Tags and Outlinks Do Not Match

This is expected. A Tags counts every rendered <a> element. Outlinks counts qualifying internal HTTP or HTTPS content-link occurrences and excludes email, telephone, JavaScript, same-page fragment, empty and static-resource destinations.

External qualifying anchors are counted in External Outlinks. See How Link Metrics Are Calculated.

Inlinks Change During the Crawl

This is expected. Inlinks is recalculated as later source pages reveal occurrences pointing to an already known target. Treat link totals as final only after the crawl has completed or stopped.

Link Counts Differ From Another Tool

Confirm whether both tools use the same:

• rendered or source HTML;
• User Agent and cookies;
• host/subdomain scope;
• fragment and query-parameter normalisation;
• treatment of repeated links;
• treatment of assets and non-HTTP schemes;
• JavaScript wait time;
• crawl completion state.

Open URL Details to compare individual occurrences instead of comparing summary totals alone.

External Links Show 000

000 means the app has no genuine HTTP status for that row. It appears before Check headers and remains when the request fails before receiving an HTTP response.

Possible causes include:

• DNS failure;
• TLS or certificate failure;
• connection refusal;
• ten-second timeout;
• firewall, rate limit or anti-bot block;
• invalid or unsupported destination;
• the check was stopped before reaching the URL.

Open URL Details and review Error and Checked At. Do not replace 000 with a made-up 4xx or 5xx code; those codes can only be reported when a server actually returns them.

External Status Differs From a Browser

External checks send HEAD first and download headers only. Some services respond differently to HEAD, to the crawler User Agent, or to automated traffic. The app retries with GET only when HEAD returns 405 or 501.

Redirect responses are intentionally retained and not followed through the external site. A 301 or 302 can therefore be correct even when a browser eventually displays a 200 destination page.

Status or Content Fields Are Blank

A blank value can be valid:

• a network failure may have no HTTP status;
• an asset has no HTML title, description or headings;
• a server may omit Last-Modified, Content-Length or language headers;
• a page may fail before rendered extraction;
• Chromium may not expose timing or protocol details for that request.

Use Content Type, Status, Error and the Headers tab together when diagnosing the row.

The Results Table Looks Empty

• Confirm that the crawl has produced at least one Crawled row.
• Clear the Filter results field.
• Open Columns and restore the default column selection.
• Switch between Internal Links and External Links.
• External Links remains empty when no qualifying off-host anchors have been captured.

Column visibility and width settings persist between launches. Settings from removed columns are ignored safely.

CSV Does Not Contain a Column

CSV export uses the currently visible Internal Links columns. Open Columns and enable the field before exporting.

The current CSV export does not include External Links or URL Details occurrence tables.

Previous Crawl Results Disappeared

The app has one working crawl database. Starting a new crawl resets it. Crawl history in the address bar records that a site was crawled, but it is not an archive of the result rows.

Use CSV export before starting the next crawl when results need to be retained.

Registration Is Not Recognised

1. Check the internet connection.
2. Open Help > Register... or Help > Registration Details....
3. Confirm that the key was entered exactly as supplied.
4. Retry validation.

A confirmed Pro registration is stored locally. A temporary validation-service failure should preserve the last confirmed registration, but an invalid, expired or revoked key may not unlock Pro modes.

Balanced or Fast Is Unavailable

Balanced and Fast are Pro crawl speeds. Confirm that the app title shows LWF SEO Web Spider Pro and review Registration Details. Standard mode includes Gentle speed.

The Licence Window Appears Again

The app asks for acceptance when the agreement version changes or when the saved acceptance setting is unavailable. Review the current terms before accepting.

Reporting a Problem

Before reporting a reproducible fault, record:

• app version and operating system;
• Standard or Pro mode;
• starting URL and selected speed;
• whether the issue occurs in Browser, Page Analysis or Crawl Results;
• the relevant Status, Error and response headers;
• the smallest URL that demonstrates the issue;
• whether the same result occurs after restarting the app.

Do not send private crawl databases or exports without checking them for credentials, personal information and sensitive URLs.