Most Hashless APIs that return a collection (a list of items) are paginated.

Paginated APIs at Hashless use the "opaque cursor" pattern. When you retrieve a paginated collection, a special next_page_token field will be included in the response. For example, when accessing any API endpoint that returns a list of transactions:

{
  "transactions": [ ... ], //... A list of up to 100 transaction objects
  "next_page_token": "dGltZT0xNjc0MjU0OTc1MDAwMTEwMDAw"
}

The value of next_page_token can be passed to a follow-up API request in the page_token query parameter. This will retrieve the next batch of collection items according to the collection's sort order -- for transactions, this means the next 100 oldest transactions.

Note that all request parameters must remain the same for the cursor to function -- taking a next_page_token from a list of transactions retrieved for address and supplying it as the page_token in a request for a different address's transactions won't work.

Cursor Stability and Incremental Pagination

The cursors returned by the Hashless API are stable, meaning that when you specify a page_token with the same value for a request with the same filtering parameters, you will always get the same items in the response.

This means you can safely store page tokens for a reasonable period in order to pause and resume a process that is "paginating through" API results -- for instance, to maximize efficiency by running your API ingestion code when resources are underutilized, or using a work queue to manage API request throughput -- without retrieving duplicate items or missing items.

Getting Results for a Specific Time Period

The Hashless Transaction APIs also accept start_time and end_time parameters to filter results to a specific time range. Prefer specifying these parameters if you are trying to retrieve transactions only within that range over paginating through the whole, unfiltered result set -- it's much faster.

You can still paginate through a result set filtered by time range if there are more than 100 results for the range.

Using Watermarks to Retrieve Only New Results

Because the Hashless API returns results in reverse chronological (newest first) order, by paginating and using the start_time filter parameter, you can effectively implement "watermarking" to retrieve only results that haven't previously been retrieved.

To start with, determine the timestamp of the most recent transaction you have retrieved from the Hashless API for a particular filter set. You could do this by querying your database for MAX(storedtimestamp)_if you store results for each set of filter parameters separately, or otherwise you can update this timestamp each time you retrieve the filter by updating a separately stored watermark.

Convert the stored timestamp into a UNIX timestamp with millisecond precision. (Hashless returns timestamps as millisecond-precision UNIX timestamps by default, so if you're storing the raw value of a Hashless timestamp field, you can use that as-is.) Use this as the value of the start_time parameter.

Retrieve the results. If you are storing watermarks separately, store the timestamp value of the first result returned in a temporary location. Proceed to paginate through the results using the page_token values, performing any processing or storage of the results you require, until the API returns an empty collection (e.g. transactions: \[]\) indicating that there are no more results.Use the temporarily-stored timestamp of the first retrieved transaction to update your stored watermark.

🚧

It's important that you update your permanently stored watermark after paginating through the whole filtered result set as opposed to after the first page of results. Otherwise, if you encounter an error while processing a subsequent page, you may "lose track" of any results appearing on later pages.

If you are retrieving an extremely high volume of transactions, you may wish to implement a more sophisticated "gap filling" system for ensuring retrieval of the full range of results across multiple time ranges. If your application needs to retrieve historical data at high scale, reach out to us for options around bulk data retrieval.