> ## Documentation Index
> Fetch the complete documentation index at: https://docs.truscan.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Read a crawl and its pages

> Returns the job and a window of its pages. `next` is the offset to ask for to continue; it is absent once the window reaches the end.



## OpenAPI

````yaml /api-reference/crawl.json get /api/crawl/{id}
openapi: 3.1.0
info:
  title: Truscan Crawl
  version: 1.0.0
  description: >-
    Walking a website: what is on it, and all of it.


    `/api/crawl/map` answers the first question and returns synchronously,
    because discovery is a handful of fetches rather than hundreds: a site that
    publishes an accurate sitemap costs one request to map.


    `/api/crawl` answers the second and returns a job, because a crawl of two
    hundred pages takes minutes and no HTTP client should hold a connection open
    for that long. Poll the job by id.


    Both respect `robots.txt`, including its crawl delay. This is not politeness
    for its own sake: every request the platform makes leaves from one address,
    so a crawler that ignores a site's wishes does not get that site blocked, it
    gets search and extract blocked for everybody.


    Results are kept for seven days and then deleted. That window is a TTL on
    the store rather than a policy document, so nothing outlives it by being
    forgotten about.
servers:
  - url: https://api.truscan.co
security:
  - bearerAuth: []
paths:
  /api/crawl/{id}:
    parameters:
      - name: id
        in: path
        required: true
        schema:
          type: string
          examples:
            - crwl_8d3ddecc038c10d4
    get:
      summary: Read a crawl and its pages
      description: >-
        Returns the job and a window of its pages. `next` is the offset to ask
        for to continue; it is absent once the window reaches the end.
      operationId: getCrawl
      parameters:
        - name: offset
          in: query
          schema:
            type: integer
            minimum: 0
            default: 0
        - name: limit
          in: query
          schema:
            type: integer
            minimum: 1
            maximum: 100
            default: 100
      responses:
        '200':
          description: The job, and a page of its results
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ResultEnvelope'
        '401':
          $ref: '#/components/responses/Failure'
        '404':
          description: >-
            No crawl with that id. A crawl that finished more than seven days
            ago has been deleted, and is reported the same way as one that never
            existed.
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Failure'
components:
  schemas:
    ResultEnvelope:
      type: object
      properties:
        success:
          type: boolean
          const: true
        message:
          type: string
        result:
          allOf:
            - $ref: '#/components/schemas/Job'
            - type: object
              properties:
                pages:
                  type: array
                  items:
                    $ref: '#/components/schemas/Page'
                next:
                  type: integer
                  description: >-
                    The offset to ask for to continue reading. Absent once the
                    window reaches the end.
    Failure:
      type: object
      properties:
        success:
          type: boolean
          const: false
        message:
          type: string
        result:
          type: object
          properties:
            code:
              type: string
    Job:
      type: object
      properties:
        id:
          type: string
          examples:
            - crwl_8d3ddecc038c10d4
        status:
          type: string
          enum:
            - queued
            - running
            - completed
            - failed
            - cancelled
        url:
          type: string
        discovered:
          type: integer
          description: >-
            URLs the crawl found. Exceeds `completed` when the page budget
            stopped it reading them all, which is the number that tells you to
            ask for more.
        completed:
          type: integer
          description: Pages read successfully. This is what is billed.
        failed:
          type: integer
          description: Pages that could not be read. Not billed.
        max_pages:
          type: integer
        created_at:
          type: string
          format: date-time
        updated_at:
          type: string
          format: date-time
        finished_at:
          type: string
          format: date-time
        expires_at:
          type: string
          format: date-time
          description: >-
            When this crawl and its pages are deleted. Reset on every write, so
            a long crawl cannot expire while it is still running.
        error:
          type: string
          description: Why a failed crawl stopped, in a sentence you can act on.
    Page:
      type: object
      properties:
        url:
          type: string
        depth:
          type: integer
          description: >-
            Links followed from the starting URL to reach this page. The seed is
            zero.
        content:
          type: object
          description: >-
            Whatever extraction produced for this page: the same shape
            /api/extract returns, including `kind`, `text`, `markdown`, and
            `tables` or `structured` when asked for.
          additionalProperties: true
        error:
          type: string
          description: >-
            Why this page has no content. Present instead of failing the crawl:
            one unreachable page out of fifty must not lose the other forty
            nine.
  responses:
    Failure:
      description: >-
        The envelope with success false. result carries the machine-readable
        code and nothing else; the HTTP status line carries the real status.
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Failure'
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: >-
        An API key, sent as `Authorization: Bearer enc_…`. A dashboard session
        cookie works on the same route.

````