Rate Limit
The public API for our Faria range of products are a shared resource, intended to be highly available as a general-purpose tool that can be called upon at any time. It is possible for client applications to fetch data throughout the day, but we ask clients to be conservative when setting up recurring processes that utilize the public APIs. As explained below, there are methods that we deploy to protect resources from overuse, and clients can react to these methods dynamically.
One method to protect resources common with API resources is rate limiting, which is a standard practice. The rate limitations may be adjusted at any time without warning, in order to ensure reliability.
The rate limit is applied per IP and accumulated regardless of endpoint. This means that if you download from one endpoint, and then start downloading at a different endpoint, the quota is accumulated for each call to each endpoint, by IP address.
Any application ingesting data from these APIs should take the following mitigation strategies to dynamically respond to any changes in rate limitations:
-
A http response of
429means that the rate limit has been exceeded. Clients may use the response headers to determine how long to wait until trying again. See table below.
-
Use the
per_pagequery parameter to set it to a reasonable value, such as50. Please note that the API may return a number of records less than passed in the request. The application code can use themetaobject to determine how many records are being returned per request.
429 Headers
When a
429 response is returned, clients may use the headers to re-try. The current rate limit can be calculated via X-RateLimit-Limit divided by X-RateLimit-Period. The below example shows the values for a rate limite of 50 requests per second (per IP):| Header Name | Description | Example |
| X-RateLimit-Limit | The number of requests that can be sent in the period indicated by X-RateLimit-Period |
50 |
| X-RateLimit-Period | The denominator of the rate limit, in seconds | 1 |
| X-RateLimit-Reset | The time when a request can be re-tried. |
2021-10-28 01:29:40 UTC |
Best Practices
In order to ensure high availability, schools can consider the following to mitigate potential for throttling:
-
In consultations with schools, there is general consensus that overnight processes are adequate in most use cases. If refreshes are needed more often, it is rarely necessary to conduct updates more often than once an hour. Please refrain from setting up recurring processes that consume 1000s of records more than necessary.
-
Use polling to repeatedly ask a particular endpoint for updates. By utilizing the
modified_sinceor `since_date` parameter whose value is set to the YYY-mm-dd time when last queried. With this method, your client application can retrieve iterative updates, ensuring that only the records which have been updated since that date passed are returned, rather than "all" the records. This results in more performant code and uses far fewer server resources.
- Not all endpoints support polling, which can be found in the reference documentation. Polling is supported if the endpoint parameters include a filter that fetches records that have been updated since the date provided.