Skip to the content

Scraper API

How the Scraper API works

You start a run with a scraper and an input. EnrichedNow reads the public pages, and the run ends with a status and a set of records. Two API calls cover it: one to start the run, one to read what it found.

A run, from request to records

  1. You start a run

    A scraper's id and its input: a domain, a search, a web address.

  2. We read the pages

    The public pages, as a visitor who is not signed in sees them.

  3. The run ends

    With a status, a reason code when it is not ok, and what it cost.

  4. You read the records

    Structured records, in pages: one per job opening, company or page.

A run can be started and left to finish, or the first call can wait up to 60 seconds and answer with the finished run. The overview shows both calls with their responses, and the documentation has the rest of the API.

Five ways a run can end

A run never ends in silence. Its status says what happened, and when it is not ok a reason code says why, such as render_required or unknown_board.

  • ok

    The run read its target and returned what it found.

    Charged

  • partial

    Some records came back and something went wrong on the way. The reason code says what.

    Charged

  • failed

    We could not read the target. The reason code says why.

    Free

  • unsupported

    The target needs something we do not do yet, such as a JavaScript render, or is a board we do not recognise.

    Free

  • cancelled

    You stopped the run.

    Free

How units are counted

What data we collect

Public data only
We read what a site shows every visitor: careers sites, job boards, LinkedIn's public pages, and the web page you name.
No logged-in pages
We never sign in to a site, hold accounts on it or use anyone's session. If a page needs a login, we do not read it.
No JavaScript rendering yet
No scraper runs a page's JavaScript. A careers page that lists its openings only that way is not read: the run ends as unsupported and is free.
No guessed values
A field we could not read is null. We do not fill it with an empty string, a zero or a likely value.

Each scraper also has limits of its own, and the line-up lists them.