Valuein
Bulk Data API

Bulk Data API Reference

Parquet-first edge gateway for bulk financial data. All endpoints stream ZSTD-compressed Parquet files — load directly into DuckDB, Pandas, or Polars.

https://data.valuein.biz
Live

Authentication

Authenticated endpoints require a Bearer token in the Authorization header. Tokens are provisioned automatically when you subscribe via Stripe. The /v1/sample/* endpoints are always public — no token required.

Unauthenticated (sample tier)

bash
curl https://data.valuein.biz/v1/sample/entity \    --output entity.parquet

Authenticated (sp500 / pro / full tier)

bash
curl https://data.valuein.biz/v1/sp500/fact \    -H "Authorization: Bearer $VALUEIN_TOKEN" \    --output fact.parquet

Set the token once: export VALUEIN_TOKEN=your_token. The Python SDK reads VALUEIN_API_KEY instead.

bash
# One company's rows only (10-digit CIK, .parquet suffix on partitions)curl https://data.valuein.biz/v1/pro/fact/0000320193.parquet \    -H "Authorization: Bearer $VALUEIN_TOKEN" \    --output aapl_fact.parquet

A request to /v1/sp500, /v1/pro or /v1/full with no Authorization header is answered with a 302 to the matching /v1/sample/{table}. Whole tables use the bare name (no .parquet); a per-CIK partition ends in /{cik}.parquet.

Check your token plan

bash
curl https://data.valuein.biz/v1/me \    -H "Authorization: Bearer $VALUEIN_TOKEN"
json
{  "plan":      "sp500",  "status":    "active",  "email":     "user@example.com",  "createdAt": "2026-04-11T00:00:00.000Z"}

Response Format

Data endpoints return raw Parquet bytes. The Content-Type is application/octet-stream. Files are ZSTD-compressed — DuckDB, Pandas, and Polars decompress automatically via read_parquet(). Non-data endpoints return application/json.

Response TypeContent-TypeEndpoints
Parquet streamapplication/octet-stream/v1/sample/*, /v1/sp500/*, /v1/pro/*, /v1/full/*
JSONapplication/json/health, /v1/me, /v1/manifest, /v1/usage

Plans

Your token's plan determines which bucket you can access. A higher plan grants access to all lower tiers as well.

PlanAuth RequiredBucketCoverage
sample
NoR2_SAMPLEPublic 5-year S&P 500 slice
sp500
YesR2_SP500Benchmark tier: the large-cap US benchmark universe incl. former members, fundamentals 1993–present
pro
YesR2_PROActive + delisted US universe (28,000+ entities), 15-year history (2011→present)
full
YesR2_FULLInstitutional tier: US + foreign issuers, 1993→present, intraday accepted_at, webhooks, internal-use license

Endpoints

12 endpoints across the gateway. Expand a row for its query parameters and an example response.

GET/health
Public
System

Liveness check. Returns 200 OK with service status.

Example responsejson
{ "ok": true, "worker": "edge-gateway", "env": "production" }
GET/v1/me
Bearer token
Auth

Returns token metadata: plan, status, email and createdAt.

Example responsejson
{ "plan": "sp500", "status": "active", "email": "user@example.com", "createdAt": "2026-04-11T00:00:00.000Z" }
GET/v1/manifest
Bearer token
Discovery

Returns available tables and last snapshot timestamp for your plan tier.

Example responsejson
{ "snapshot": "snapshot_20260411", "last_updated": "2026-04-11T00:00:00Z", "tables": [...] }
GET/v1/sample/manifest
Public
Sample (No Auth)

Public sample tier manifest — no token required. Includes the table list, the column schema and an upgrade message.

Example responsejson
{ "snapshot": "snapshot_20260411", "tables": [...], "upgrade": "…" }
GET/v1/sample/{table}
Public
Sample (No Auth)

Parquet stream from the public sample bucket (5-year S&P 500 slice). No token required. Use the bare table name with no .parquet suffix (a suffix on a monolithic table returns 400). The tables served are listed in /v1/sample/manifest.

Example responsejson
application/octet-stream — raw Parquet bytes
GET/v1/sp500/{table}
Bearer token
Data

Parquet stream for the Benchmark universe (current and former S&P 500 constituents). Requires a Benchmark (sp500), Pro, or Institutional token. Full history, 500+ tickers. Without an Authorization header the gateway answers 302 and redirects to /v1/sample/{table}.

Example responsejson
application/octet-stream — raw Parquet bytes
GET/v1/pro/{table}
Bearer token
Data

Parquet stream from the Pro bucket. Requires pro or full plan token. Full active + delisted US universe (28,000+ entities), 15-year rolling history (2011→present). Without an Authorization header the gateway answers 302 and redirects to /v1/sample/{table}.

Example responsejson
application/octet-stream — raw Parquet bytes
GET/v1/full/{table}
Bearer token
Data

Parquet stream from the Institutional bucket. Requires `full` plan token. 28,000+ US-listed entities plus foreign issuers, active + delisted, 1993→present. Without an Authorization header the gateway answers 302 and redirects to /v1/sample/{table}.

Example responsejson
application/octet-stream — raw Parquet bytes
GET/v1/{plan}/{table}/{cik}.parquet
Bearer token
Data

One company's rows of a per-CIK partitioned table (fact, ratio, filing, earnings_signals, stock_price, stock_price_daily; Institutional adds insider_transaction, insider_ownership and institutional_holding). {plan} is sp500, pro or full; {cik} is the 10-digit zero-padded CIK. Much smaller than the whole table. institutional_holding also has /v1/full/institutional_holding/by_issuer/{cik}.parquet and /by_filer/{cik}.parquet.

Example responsejson
application/octet-stream — raw Parquet bytes
GET/v1/presign/{table}
Bearer token
Data

Returns a short-lived signed URL for a table (or /v1/presign/{table}/{cik}.parquet for one company's partition), so a client such as DuckDB can read it directly. Expiry is capped by rateLimits.presignMaxExpiry.

Example responsejson
{ "url": "https://…", "expires_in": 3600 }
GET/v1/plans
Public
Discovery

Plans, tiers and rate limits. The canonical source for the numbers on this page.

Example responsejson
{ "version": "…", "plans": { "sample": {…}, "sp500": {…}, "pro": {…}, "full": {…} } }
GET/v1/usage
Bearer token
Analytics

Returns daily API call counts, error rates, and per-table breakdowns for the last N days (default 7, max 30).

Query Parameters

daysNumber of days to return (1–30). Defaults to 7.
Example responsejson
{ "period_days": 7, "total_calls": 1420, "error_rate": 0.012, "daily": [...], "table_breakdown": {...} }

Python Example

Download a Parquet table and query it locally with DuckDB in under 10 lines.

quickstart.pypython
# pip install duckdb requestsimport os import duckdbimport requests token = os.environ["VALUEIN_TOKEN"]  # your Bearer tokenurl   = "https://data.valuein.biz/v1/sp500/fact" r = requests.get(url, headers={"Authorization": f"Bearer {token}"}, stream=True)r.raise_for_status() with open("fact.parquet", "wb") as f:    for chunk in r.iter_content(chunk_size=8192):        f.write(chunk) conn = duckdb.connect()df   = conn.execute(    "SELECT * FROM read_parquet('fact.parquet') LIMIT 5").df()print(df)

Available Tables

13 core tables ship on every tier. Pass any table name as the {table} path segment. Institutional adds 11 more (insider Forms 3/4/5/144, 13F/13D/13G holdings and Form ADV), listed below. See Parquet Schema Reference for full field definitions.

TableDescription
referencesDerived flat join of entity + security. One row per security. Start here for cross-company queries; JOIN index_membership on cik = cik for membership filters.
entityCompany profiles: name, sector, SIC code, location, CEO, founding year, description. One row per CIK.
securityExchange listings: ticker, exchange, FIGI, valid date range (SCD Type 2). Multiple rows per company.
filingSEC EDGAR filing index: accession ID, form type, filing date, acceptance timestamp. Links entity to facts.
fact120.5M+ financial data points: XBRL concept values with accepted_at timestamps for PIT accuracy.
ratioPipeline-computed financial ratios per entity per fiscal period. Carries an append-only accepted_at vintage, so ratios are point-in-time reconstructable like fundamentals. Filter by category for grouped screens.
factor_scoresCross-sectional factor scores (value, quality, momentum families) per entity — powers universe screens.
earnings_signalsPer-entity earnings signals: surprises, revision momentum, and report-timing features.
stock_priceCoarse price series per entity, available on every tier: one close at each fiscal period end plus month-end closes, each carrying the priced listing's total_return_index — so the ratio of two month-end rows is the total return between them, dividends included.
stock_price_dailyPro and Institutional only — the free tiers do not carry the daily series (licensed market data). Full daily OHLCV bar series with total_return_index and the raw corporate-action factors — join to fundamentals for PIT valuation multiples. Use total_return_index for total return; adjusted_close is a sparse vendor passthrough. Prices are licensed market data, not EDGAR: no bars in 1993, earliest bar is 1994 or later and differs per security.
index_membershipHistorical index constituents (SP500, RUSSELL1000, RUSSELL2000, RUSSELL3000) with effective_date / removal_date for PIT universe construction. Keys on cik.
taxonomy_guideMapping of 350+ standard_concept labels to raw XBRL tags and human-readable descriptions.
standard_conceptThe standardization layer itself: every canonical concept with definition, statement type, and review status (rule-based mappings checked by accounting identities).

Paid-tier and Institutional tables

Not in the free sample. Institutional tables answer on /v1/full/ only; tables marked monolithic have no per-CIK partitions.

TableTierDescription
filing_registrantPaid tiersEvery registrant of an SEC accession. filing has one row per accession, owned by the first registrant, so a combined (co-registrant) filing appears under one CIK there. Join to filing on accession_id to list a company's filings including the combined ones. Not in the free sample manifest.
insider_partyInstitutionalDirectory of every insider or reporting person on Forms 3/4/5/144. Joins to insider_transaction via insider_party_id.
insider_filingInstitutionalForm 3/4/5, SC 13D/13G and 144 disclosure filings (and amendments), rooted on the issuer in subject_cik.
insider_transactionInstitutionalForm 3/4/5/144 line items: one transaction, initial holding or proposed sale per row. Also served per CIK.
insider_ownershipInstitutionalSC 13D / 13G beneficial-ownership disclosures (5%+ stakes), one row per filing and reporting person. Also served per CIK.
insider_footnoteInstitutionalForm 3/4/5 footnotes, verbatim: the receipt behind a 10b5-1 plan or a weighted-average price. Joins to insider_filing on accession_id. Monolithic file.
insider_ownership_purposeInstitutionalSchedule 13D Item 4 Purpose of Transaction, verbatim raw text. 13D and 13D/A only. Joins to insider_filing on accession_id. Monolithic file.
institutional_filingInstitutionalForm 13F-HR / 13F-NT filings (and amendments), rooted on the manager.
institutional_holdingInstitutionalForm 13F-HR holdings: one row per filed information-table line. Also served per CIK, by issuer or by filer.
institutional_other_managerInstitutional13F manager lists: other included managers and managers reporting for the filer. Joins to institutional_filing on accession_id. Monolithic file.
investment_adviserInstitutionalForm ADV Part 1A advisers, keyed on crd_number (cik is a nullable soft link). Exempt Reporting Advisers omit Items 5.A to 5.F by obligation.
investment_adviser_private_fundInstitutionalPrivate funds advised by a registered adviser (Form ADV Schedule D 7.B.1). Joins to investment_adviser on crd_number.

Manifest Response

Call GET /v1/manifest to discover available tables and the current snapshot timestamp for your plan. Check this before downloading tables to detect updates.

json
{  "snapshot":     "snapshot_YYYYMMDD",  "last_updated": "YYYY-MM-DDTHH:MM:SSZ",  "tables": ["earnings_signals", "entity", "fact", "factor_scores",             "filing", "filing_registrant", "index_membership",             "industry_classification", "ratio", "references",             "restatement_events", "security", "standard_concept",             "stock_price", "taxonomy_guide"]}

Rate Limits & Retries

Limits are per Bearer token and enforced at the Cloudflare edge. Every response includes the standard rate-limit headers — read them before retrying so your client never busy-waits.

Response headers on every request

http
X-RateLimit-Limit:     120
X-RateLimit-Remaining: 117
X-RateLimit-Reset:     1735680000
Retry-After:           42

Retry-After appears only on 429 responses (seconds). X-RateLimit-Reset is the Unix epoch when the window rolls over.

Recommended retry policy

  • 429 — sleep for Retry-After seconds, then retry. Never retry sooner.
  • 503 — exponential backoff: 1s, 2s, 4s. Cap at 3 retries.
  • 5xx other — retry once after 1s, then surface the error.
  • 4xx other — never retry. Fix the request.

curl with retry-aware streaming

bash
# --retry honors Retry-After on 429 and exponentially backs off on 5xx.# --retry-max-time bounds total elapsed time; --retry-connrefused covers# the cold-start case after a cache eviction.curl https://data.valuein.biz/v1/sp500/fact \    -H "Authorization: Bearer $VALUEIN_TOKEN" \    --retry 5 --retry-max-time 120 \    --retry-connrefused \    --output fact.parquet

Parquet responses are a single binary stream — no pagination, no cursor tokens. The gateway sets Content-Length on every data response so clients can show progress and pre-allocate buffers. For incremental reads, query the latest snapshot from /v1/manifest and download only when the timestamp changes.

Bulk Data API vs MCP Server vs Python SDK

Three channels, one Bearer token, same warehouse. Pick by access pattern.

ChannelBest forReturnsLatency
Bulk Data APILoading full tables into DuckDB, Spark, or a warehouse. Periodic syncs.ZSTD Parquet (full table)Edge stream · MB-scale
MCP ServerSingle-fact lookups from AI agents. Conversational queries.JSON tool responsesSub-100ms typical
Python SDKDataFrame-shaped queries from notebooks and scripts.DuckDB over R2 Parquet · as-filed vs latest restatement columnsLocal DuckDB · ms

Use the SDK first if you're writing Python — it wraps this API and the MCP server with sensible defaults. Use the raw Bulk Data API when you're in a non-Python stack or building a partner integration.

Error Codes

StatusMeaningCommon Cause
200 OKSuccessRequest succeeded. Parquet bytes or JSON body in the response.
400 Bad RequestInvalid tableThe table name in the path is not in the valid tables list. Check spelling and trailing slashes.
401 UnauthorizedMissing or invalid tokenNo Authorization header, malformed Bearer token, or token not found in KV store.
403 ForbiddenPlan too lowYour token exists but its plan does not grant access to this bucket (e.g. sample token accessing /v1/sp500/).
429 Too Many RequestsRate limit exceededYou have exceeded your per-minute or per-hour rate limit, or your plan's daily bulk-download limit (resets at UTC midnight). Slow down and retry after the Retry-After header; a higher plan has higher limits.
503 Service UnavailableSnapshot loadingThe R2 snapshot is being refreshed. Retry after 30–60 seconds. This is rare and brief.

Get your API token

Subscribe to the free Benchmark plan, Pro, or Institutional to receive a Bearer token instantly. The sample tier is always free — no credit card required.

Index names are trademarks of their owners; Valuein is not affiliated with or endorsed by any index provider.