Published datasets and pagination

Use the same list/detail pattern for every collection, including organizations, agencies (filtered organizations), programs, awards and relationship data.

Discover the schema

GET /api/v1/datasets returns data and build_id. Each catalog entry contains:

Fields are an explicit reviewed projection. New database columns are not automatically exposed. Some expensive collections are unavailable; the runtime catalog decides access.

List and search

GET /api/v1/datasets/{dataset} returns {data, next_cursor, build_id}. Each item is {record_id, attributes}. A positional page may also have truncated.

q searches whichever of name, title, description, summary and recipient_name exist in the dataset. If none exists, text search returns 400. It does not search aliases, codes or every string field. It provides no fuzzy matching, relevance ranking or custom ordering.

Exact filters use AND. Supported values are a string (up to 500 characters), number, boolean or null. Null means SQL IS NULL. Use strings for precise large integers and decimals. Array containment, nested JSON predicates, OR, ranges, field selection and arbitrary sorting are not supported. Unknown filter fields return 400.

curl --fail-with-body --get \
  'https://app.defdex.io/api/v1/datasets/organizations' \
  --header "Authorization: Bearer $DEFDEX_API_KEY" \
  --data-urlencode 'filters={"type":"agency","sub_type":"department"}' \
  --data-urlencode 'limit=25'

Every database read has a five-second query budget. A query-budget 400 means narrow with exact filters; repeating the same broad request is unlikely to help. In particular, summary-award-actions requires award_key, and summary-fy-totals requires fiscal_year. See each collection’s required_filters.

Follow pagination

Continue with the same dataset, q and filters; add the returned next_cursor. Do not decode, edit or construct a cursor. Save a cursor only after processing its successful page. Never advance after a failed request.

from defdex import DefDexClient
api = DefDexClient()
cursor = None
while True:
    page = api.list("organizations", filters={"type": "agency"},
                    limit=100, cursor=cursor)
    for record in page["data"]:
        print(record["attributes"]["id"], record["attributes"]["name"])
    if page.get("truncated"):
        raise RuntimeError("Query cap reached: narrow exact filters before continuing")
    cursor = page["next_cursor"]
    if cursor is None:
        break

This example stops on an exception; use the bounded retry policy in errors for production workloads. Make requests sequentially and respect the shared account limit.

Know which identity you are using

Positional datasets expose at most 10,000 positions per query. If more matches exist, the final accessible page has truncated: true and next_cursor: null. A null cursor in that case does not mean a complete export. Narrow exact filters; there is no bulk-export endpoint.

A stable-key traversal is not a point-in-time snapshot across a publication change. Compare build_id across pages and related reads. Restart if your task requires one consistent build. The API rejects a build change during an individual read. Award convenience cursors are build-bound and return 409 after a publication change.

Read a single record

GET /datasets/{dataset}/{record_id} returns {data, build_id}. Use the ID returned by the same collection, encoded as one path segment. attributes.id, an external award key and a website slug are not interchangeable with record_id.

Use an exact id or slug filter to resolve an entity first. The SDKs encode detail IDs automatically. Retain both the entity key and opaque record ID when they serve different purposes in your application.