Pagination and Large Requests
Details
Docs navigation
- Getting started: Quickstart, Authentication, Rate limits, Errors and retries
- Guides: Symbol formats, Timestamps and time zones, Pagination (you are here), Split and dividend adjustments, WebSocket streaming, Dataset downloads, Excel and Google Sheets, Changelog
- API reference: Stock bars, Stock trades and quotes, Snapshots and movers, Tickers, exchanges, calendar, Splits and dividends, Fundamentals and SEC filings, Options chain and Greeks, Options EOD, bars, OI, Forex, Crypto, Crypto derivatives, Futures bars and continuous, Settlements and COT, Indices, Economic data, Technical indicators
- SDKs: Python, JavaScript
Ten years of 1-minute bars for one stock is close to a million rows, and a busy day of trades can be several million. No API returns that in one response. This page explains the cursor-based API pagination we use, the page size limits, and the patterns that work for very large pulls, including when to stop paging and buy a file instead.
Early access. Describes the planned v1 API.
How it works
How paging works
1. Make the first request
Ask for the range you want. Add limit to set the page size.
Example (curl):
curl -H "Authorization: Bearer YOUR_API_KEY" \
"https://api.marketdata.co/v1/stocks/bars?symbol=NVDA&interval=1m&start=2026-01-01&end=2026-07-01&limit=50000"
2. Check next_cursor
{
"status": "ok",
"count": 50000,
"next_cursor": "eyJ0IjoiMjAyNi0wMy0xMlQxNTo0Mjo",
"results": [ ... ]
}
If next_cursor is a string, there is more data. A null means you have everything.
3. Request the next page
Send the same request with cursor set to that value. Keep the other parameters identical.
Example (curl):
curl -H "Authorization: Bearer YOUR_API_KEY" \
"https://api.marketdata.co/v1/stocks/bars?symbol=NVDA&interval=1m&start=2026-01-01&end=2026-07-01&limit=50000&cursor=eyJ0IjoiMjAyNi0wMy0xMlQxNTo0Mjo"
4. Loop until done
Example (Python):
import os
import requests
import pandas as pd
API = "https://api.marketdata.co/v1"
H = {"Authorization": f"Bearer {os.environ['MARKETDATA_API_KEY']}"}
def fetch_all(path, params):
params = dict(params, limit=50000)
frames = []
while True:
r = requests.get(f"{API}{path}", headers=H, params=params, timeout=60)
r.raise_for_status()
body = r.json()
frames.append(pd.DataFrame(body["results"]))
if body["next_cursor"] is None:
break
params["cursor"] = body["next_cursor"]
return pd.concat(frames, ignore_index=True)
nvda = fetch_all("/stocks/bars", {"symbol": "NVDA", "interval": "1m", "start": "2026-01-01", "end": "2026-07-01"})
Details
Page sizes
| Endpoint type | Default limit |
Maximum limit |
|---|---|---|
| Bars (stocks, options, FX, crypto, futures, indices) | 5,000 | 50,000 |
| Trades and quotes (ticks) | 5,000 | 50,000 |
| Reference lists (tickers, contracts) | 1,000 | 10,000 |
| Corporate actions, filings, calendars | 1,000 | 10,000 |
Results are sorted by time, oldest first. Pass sort=desc for newest first; the cursor follows the same order.
Details
Patterns for big pulls
Split by time as well as by page. For tick data, request one trading day per call and page within it. If something fails, you redo one day, not a month.
Run symbols in parallel, pages in sequence. A cursor depends on the page before it, so pages for one query must run in order. Different symbols can run side by side, up to your rate limit.
Write each page to disk. Append each page to a Parquet or CSV file as it arrives. A crash at page 37 should not lose pages 1 to 36.
Count your calls first. Rows divided by page size is the number of requests. One year of 1-minute bars for one stock is about 98,000 regular-session rows, or two pages at 50,000. A year of trades for a large stock can run to hundreds of pages.
Consider a file. If you need the same history for hundreds of symbols, a one-off dataset download is faster and cheaper. US stock 1-minute history is $5 per symbol as a file.
Watch out
Notes and gotchas
- Cursors are tied to the query that produced them. Change any parameter and you get
400 invalid_cursor. - A cursor expires after 24 hours. Restart the query if a job was paused overnight.
- A page can hold fewer rows than
limitand still have anext_cursor. Check the cursor, not the row count. range_too_largemeans one call spans too much for the interval (for example 20 years of1sbars). Split the range into smaller windows.- Each page counts as one request against your rate limit.
- CSV responses (
format=csv) return the cursor in theX-Next-Cursorheader, since the body has no room for it.
Keep reading
Next steps
Details
Was this page useful?
[Yes] [No]
Need a bulk pull we do not cover? Tell us on the custom data form.