Initial Data API documentation: manifest, list, facts, recent changes, RDF export, with example code and sync pattern (via create-page on MediaWiki MCP Server)
 
Fix all api.php URLs: $wgScriptPath is '' on this wiki, so api.php is at the document root, not /w/ (via update-page on MediaWiki MCP Server)
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
{{TOC right}}
__NOCACHE__
__TOC__
The '''EveryCRT Data API''' is a documented way to fetch the structured data of [[Main Page|EveryCRT]] programmatically — for example, to build a third-party app that mirrors, indexes, or analyzes the wiki.
The '''EveryCRT Data API''' is a documented way to fetch the structured data of [[Main Page|EveryCRT]] programmatically — for example, to build a third-party app that mirrors, indexes, or analyzes the wiki.


Line 5: Line 6:


No authentication is required. All data is publicly readable.
No authentication is required. All data is publicly readable.
== URL layout ==
This wiki uses two URL paths:
* '''Articles''' live at <code>/wiki/PageName</code> (e.g. <code>https://wiki.everycrt.com/wiki/Sony_KV-20FV300</code>).
* '''Scripts''' live at the document root: <code>/api.php</code>, <code>/rest.php</code>, <code>/index.php</code>, <code>/load.php</code>. The Action API is at <code>https://wiki.everycrt.com/api.php</code> — not <code>/w/api.php</code> as on Wikipedia.


== Sync pattern ==
== Sync pattern ==
Line 18: Line 26:


The manifest is a JSON document listing every canonical entity-type category, with live page counts and ready-to-use enumeration URLs. It updates automatically as the catalog grows; new entity types added in the future will appear here without API clients needing code changes (assuming they iterate over <code>entity_types</code> rather than hardcoding).
The manifest is a JSON document listing every canonical entity-type category, with live page counts and ready-to-use enumeration URLs. It updates automatically as the catalog grows; new entity types added in the future will appear here without API clients needing code changes (assuming they iterate over <code>entity_types</code> rather than hardcoding).
'''Human-readable view:''' [[EveryCRT:Data API/manifest]]


'''Programmatic fetch (recommended):'''
'''Programmatic fetch (recommended):'''


<pre>GET https://wiki.everycrt.com/w/api.php?action=expandtemplates&text=%7B%7B%23invoke%3ADataAPIManifest%7Cjson%7D%7D&prop=wikitext&format=json&formatversion=2</pre>
<pre>GET https://wiki.everycrt.com/api.php?action=expandtemplates&text=%7B%7B%23invoke%3ADataAPIManifest%7Cjson%7D%7D&prop=wikitext&format=json&formatversion=2</pre>


The response wraps the manifest in <code>expandtemplates.wikitext</code> as a string; parse that string as JSON:
The response wraps the manifest in <code>expandtemplates.wikitext</code> as a string; parse that string as JSON:


<syntaxhighlight lang="javascript">
<pre>
const url = 'https://wiki.everycrt.com/w/api.php'
const url = 'https://wiki.everycrt.com/api.php'
   + '?action=expandtemplates'
   + '?action=expandtemplates'
   + '&text=%7B%7B%23invoke%3ADataAPIManifest%7Cjson%7D%7D'
   + '&text=%7B%7B%23invoke%3ADataAPIManifest%7Cjson%7D%7D'
Line 40: Line 46:


console.log(manifest.entity_types);
console.log(manifest.entity_types);
</syntaxhighlight>
</pre>
 
The current manifest is rendered at the [[#Live manifest|bottom of this page]] for human reading.


== Listing entities ==
== Listing entities ==
Line 46: Line 54:
Use Semantic MediaWiki's <code>action=ask</code> endpoint to enumerate pages of a given entity type. The manifest's <code>list_url</code> for each entity type does exactly this — returning every page with its <code>Modification date</code>, which is what you want for indexing and for driving incremental sync:
Use Semantic MediaWiki's <code>action=ask</code> endpoint to enumerate pages of a given entity type. The manifest's <code>list_url</code> for each entity type does exactly this — returning every page with its <code>Modification date</code>, which is what you want for indexing and for driving incremental sync:


<pre>GET https://wiki.everycrt.com/w/api.php?action=ask&format=json&formatversion=2&query=[[Category:CRT models]]|?Modification date|limit=500</pre>
<pre>GET https://wiki.everycrt.com/api.php?action=ask&format=json&formatversion=2&query=[[Category:CRT models]]|?Modification date|limit=500</pre>


The query supports any SMW ask syntax — additional printouts, filters, ordering, etc.
The query supports any SMW ask syntax — additional printouts, filters, ordering, etc.
Line 55: Line 63:
'''Example — all Sony CRT models with screen size and release year:'''
'''Example — all Sony CRT models with screen size and release year:'''


<pre>GET https://wiki.everycrt.com/w/api.php?action=ask&format=json&query=[[Category:CRT models]][[Has brand::Sony]]|?Has screen size inches|?Has release year|limit=500</pre>
<pre>GET https://wiki.everycrt.com/api.php?action=ask&format=json&query=[[Category:CRT models]][[Has brand::Sony]]|?Has screen size inches|?Has release year|limit=500</pre>


For interactive query development, use [[Special:Ask]].
For interactive query development, use [[Special:Ask]].
Line 63: Line 71:
For a single page, the <code>action=browsebysubject</code> endpoint returns ''every'' SMW fact known about it — including subobjects (CRT inputs, outputs, voltage ranges, etc.):
For a single page, the <code>action=browsebysubject</code> endpoint returns ''every'' SMW fact known about it — including subobjects (CRT inputs, outputs, voltage ranges, etc.):


<pre>GET https://wiki.everycrt.com/w/api.php?action=browsebysubject&format=json&formatversion=2&subject=Sony_KV-20FV300</pre>
<pre>GET https://wiki.everycrt.com/api.php?action=browsebysubject&format=json&formatversion=2&subject=Sony_KV-20FV300</pre>


This is the right endpoint for "give me everything you know about this entity" — no need to enumerate properties up front. New properties added to the wiki are automatically included in the response.
This is the right endpoint for "give me everything you know about this entity" — no need to enumerate properties up front. New properties added to the wiki are automatically included in the response.
Line 75: Line 83:
For incremental sync, poll <code>action=query&list=recentchanges</code> filtered by a timestamp. The standard MediaWiki [[Special:RecentChanges]] view is the human-readable equivalent:
For incremental sync, poll <code>action=query&list=recentchanges</code> filtered by a timestamp. The standard MediaWiki [[Special:RecentChanges]] view is the human-readable equivalent:


<pre>GET https://wiki.everycrt.com/w/api.php?action=query&list=recentchanges&format=json&formatversion=2&rcprop=title|timestamp|ids|sizes|user&rclimit=500&rcend=2026-05-01T00:00:00Z</pre>
<pre>GET https://wiki.everycrt.com/api.php?action=query&list=recentchanges&format=json&formatversion=2&rcprop=title|timestamp|ids|sizes|user&rclimit=500&rcend=2026-05-01T00:00:00Z</pre>


A typical sync loop:
A typical sync loop:


# Save your "last sync" timestamp (UTC ISO 8601) after each successful pass.
# Save your "last sync" timestamp (UTC ISO 8601) after each successful pass.
# On the next sync, fetch recent-changes with <code>rcend=<previous timestamp></code>.
# On the next sync, fetch recent-changes with <code>rcend=&lt;previous timestamp&gt;</code>.
# For each unique <code>title</code> in the response, re-fetch via <code>browsebysubject</code> (or remove from your index if it was a deletion — see <code>rctype=log</code> and <code>rcshow</code> options for handling deletes and moves).
# For each unique <code>title</code> in the response, re-fetch via <code>browsebysubject</code> (or remove from your index if it was a deletion — see <code>rctype=log</code> and <code>rcshow</code> options for handling deletes and moves).
# Save the new "last sync" timestamp.
# Save the new "last sync" timestamp.
Line 86: Line 94:
To filter to specific namespaces, add <code>&rcnamespace=0|3000</code> (main + Document, for example). Namespace IDs are listed via:
To filter to specific namespaces, add <code>&rcnamespace=0|3000</code> (main + Document, for example). Namespace IDs are listed via:


<pre>GET https://wiki.everycrt.com/w/api.php?action=query&meta=siteinfo&siprop=namespaces&format=json&formatversion=2</pre>
<pre>GET https://wiki.everycrt.com/api.php?action=query&meta=siteinfo&siprop=namespaces&format=json&formatversion=2</pre>


== RDF export ==
== RDF export ==
Line 115: Line 123:


If your app is doing something cool with EveryCRT data, drop a note on the community channels — we'd love to see it.
If your app is doing something cool with EveryCRT data, drop a note on the community channels — we'd love to see it.
== Live manifest ==
This is the manifest as it would be returned by the [[#Manifest|programmatic fetch]] above, rendered live from [[Module:DataAPIManifest]]:
{{#tag:pre|{{#invoke:DataAPIManifest|json}}}}


== See also ==
== See also ==
Line 121: Line 135:
* [[Special:Ask]] — interactive SMW query builder
* [[Special:Ask]] — interactive SMW query builder
* [[Special:Browse]] — interactive per-page fact browser
* [[Special:Browse]] — interactive per-page fact browser
* [[EveryCRT:Data API/manifest]] — human-readable view of the manifest
* [[Module:DataAPIManifest]] — Lua source of the manifest
* [[Module:DataAPIManifest]] — Lua source of the manifest
* [[EveryCRT:Maintenance]] — wiki health and category audits
* [[EveryCRT:Maintenance]] — wiki health and category audits


[[Category:EveryCRT administration]]
[[Category:EveryCRT administration]]

Latest revision as of 20:17, 12 May 2026

The EveryCRT Data API is a documented way to fetch the structured data of EveryCRT programmatically — for example, to build a third-party app that mirrors, indexes, or analyzes the wiki.

It is not a custom REST service. It is a thin documentation layer over the standard MediaWiki Action API and Semantic MediaWiki endpoints, plus one wiki-maintained manifest that lists the canonical entity-type categories.

No authentication is required. All data is publicly readable.

URL layout

This wiki uses two URL paths:

Sync pattern

The recommended way to mirror the wiki is two-step:

  1. Fetch the manifest once to discover the entity types and how to enumerate each.
  2. For each entity type, fetch its list of pages (with last-modified timestamps), then call entity facts on each page you want full data for.

For ongoing incremental sync, poll recent changes since your last sync timestamp and re-fetch only the affected pages.

Manifest

The manifest is a JSON document listing every canonical entity-type category, with live page counts and ready-to-use enumeration URLs. It updates automatically as the catalog grows; new entity types added in the future will appear here without API clients needing code changes (assuming they iterate over entity_types rather than hardcoding).

Programmatic fetch (recommended):

GET https://wiki.everycrt.com/api.php?action=expandtemplates&text=%7B%7B%23invoke%3ADataAPIManifest%7Cjson%7D%7D&prop=wikitext&format=json&formatversion=2

The response wraps the manifest in expandtemplates.wikitext as a string; parse that string as JSON:

const url = 'https://wiki.everycrt.com/api.php'
  + '?action=expandtemplates'
  + '&text=%7B%7B%23invoke%3ADataAPIManifest%7Cjson%7D%7D'
  + '&prop=wikitext&format=json&formatversion=2';

const res = await fetch(url, {
  headers: { 'User-Agent': 'my-app/1.0 (https://example.com/contact)' },
});
const { expandtemplates } = await res.json();
const manifest = JSON.parse(expandtemplates.wikitext);

console.log(manifest.entity_types);

The current manifest is rendered at the bottom of this page for human reading.

Listing entities

Use Semantic MediaWiki's action=ask endpoint to enumerate pages of a given entity type. The manifest's list_url for each entity type does exactly this — returning every page with its Modification date, which is what you want for indexing and for driving incremental sync:

GET https://wiki.everycrt.com/api.php?action=ask&format=json&formatversion=2&query=[[Category:CRT models]]|?Modification date|limit=500

The query supports any SMW ask syntax — additional printouts, filters, ordering, etc.

  • Page through with |offset=N in the query.
  • Maximum limit per request is 500.

Example — all Sony CRT models with screen size and release year:

GET https://wiki.everycrt.com/api.php?action=ask&format=json&query=[[Category:CRT models]][[Has brand::Sony]]|?Has screen size inches|?Has release year|limit=500

For interactive query development, use Special:Ask.

Entity facts

For a single page, the action=browsebysubject endpoint returns every SMW fact known about it — including subobjects (CRT inputs, outputs, voltage ranges, etc.):

GET https://wiki.everycrt.com/api.php?action=browsebysubject&format=json&formatversion=2&subject=Sony_KV-20FV300

This is the right endpoint for "give me everything you know about this entity" — no need to enumerate properties up front. New properties added to the wiki are automatically included in the response.

Subobjects are returned as additional subjects with names like Sony_KV-20FV300#_HASH. Treat them as child records of the parent page; together they describe the CRT inputs, outputs, voltage ranges, and so on.

For interactive browsing, use Special:Browse.

Recent changes

For incremental sync, poll action=query&list=recentchanges filtered by a timestamp. The standard MediaWiki Special:RecentChanges view is the human-readable equivalent:

GET https://wiki.everycrt.com/api.php?action=query&list=recentchanges&format=json&formatversion=2&rcprop=title|timestamp|ids|sizes|user&rclimit=500&rcend=2026-05-01T00:00:00Z

A typical sync loop:

  1. Save your "last sync" timestamp (UTC ISO 8601) after each successful pass.
  2. On the next sync, fetch recent-changes with rcend=<previous timestamp>.
  3. For each unique title in the response, re-fetch via browsebysubject (or remove from your index if it was a deletion — see rctype=log and rcshow options for handling deletes and moves).
  4. Save the new "last sync" timestamp.

To filter to specific namespaces, add &rcnamespace=0|3000 (main + Document, for example). Namespace IDs are listed via:

GET https://wiki.everycrt.com/api.php?action=query&meta=siteinfo&siprop=namespaces&format=json&formatversion=2

RDF export

For semantic-web-style consumption, Special:ExportRDF returns RDF/OWL for individual pages or entire category subtrees:

GET https://wiki.everycrt.com/wiki/Special:ExportRDF?pages=Category:CRT%20models&recursive=1

This is more verbose than the JSON endpoints; most app developers will prefer browsebysubject.

Conventions and stability

  • All endpoints return UTF-8.
  • Page titles are case-sensitive after the first character (standard MediaWiki).
  • Subobjects (per-input, per-output, per-voltage-range facts) appear as separate subjects in browsebysubject output, with titles ending in #_HASH. Treat them as child records of the parent page; you do not need to fetch them separately.
  • The set of properties on any one entity type may grow over time. Clients using browsebysubject automatically pick up new properties; clients using fixed action=ask printouts will need updates when they want to read newly added fields.
  • Entity-type categories listed in the manifest are stable. New entity types may be added; existing ones will not be silently renamed.
  • This is a community wiki; data quality varies. Treat values as best-effort, especially for newer or less-documented models.

Etiquette

The wiki is publicly readable and no authentication is required for any of these endpoints. Please be a good citizen:

  • Set a descriptive User-Agent on your requests (e.g. my-app/1.0 (https://example.com/contact)) so the wiki operators can reach you if there's a problem.
  • Cache the manifest — it changes slowly (new entity types are rare).
  • For full syncs, sleep briefly between requests; a few requests per second is fine.
  • Use recent changes for incremental sync rather than re-walking the whole catalog every time.

If your app is doing something cool with EveryCRT data, drop a note on the community channels — we'd love to see it.

Live manifest

This is the manifest as it would be returned by the programmatic fetch above, rendered live from Module:DataAPIManifest:

{
    "version": 1,
    "entity_types": [
        {
            "list_url": "https://wiki.everycrt.com/api.php?action=ask&format=json&formatversion=2&query=%5B%5BCategory%3ACRT+models%5D%5D%7C%3FModification+date%7Climit%3D500%7Coffset%3D0",
            "count": 2152,
            "key": "crt_model",
            "label": "CRT models",
            "category_url": "https://wiki.everycrt.com/wiki/Category:CRT_models",
            "ask_query": "[[Category:CRT models]]",
            "category": "CRT models",
            "description": "CRT monitors, televisions, and professional video displays."
        },
        {
            "list_url": "https://wiki.everycrt.com/api.php?action=ask&format=json&formatversion=2&query=%5B%5BCategory%3AAV+devices%5D%5D%7C%3FModification+date%7Climit%3D500%7Coffset%3D0",
            "count": 1069,
            "key": "av_device",
            "label": "AV devices",
            "category_url": "https://wiki.everycrt.com/wiki/Category:AV_devices",
            "ask_query": "[[Category:AV devices]]",
            "category": "AV devices",
            "description": "VCRs, TBCs, video mixers, switchers, and other vintage AV equipment."
        },
        {
            "list_url": "https://wiki.everycrt.com/api.php?action=ask&format=json&formatversion=2&query=%5B%5BCategory%3AAccessories%5D%5D%7C%3FModification+date%7Climit%3D500%7Coffset%3D0",
            "count": 49,
            "key": "accessory",
            "label": "Accessories",
            "category_url": "https://wiki.everycrt.com/wiki/Category:Accessories",
            "ask_query": "[[Category:Accessories]]",
            "category": "Accessories",
            "description": "Remotes, cables, antennas, and other accessories."
        },
        {
            "list_url": "https://wiki.everycrt.com/api.php?action=ask&format=json&formatversion=2&query=%5B%5BCategory%3AParts%5D%5D%7C%3FModification+date%7Climit%3D500%7Coffset%3D0",
            "count": 13,
            "key": "part",
            "label": "Parts",
            "category_url": "https://wiki.everycrt.com/wiki/Category:Parts",
            "ask_query": "[[Category:Parts]]",
            "category": "Parts",
            "description": "ICs, transistors, CRT tubes, flyback transformers, and other components."
        },
        {
            "list_url": "https://wiki.everycrt.com/api.php?action=ask&format=json&formatversion=2&query=%5B%5BCategory%3ARepair+services%5D%5D%7C%3FModification+date%7Climit%3D500%7Coffset%3D0",
            "count": 7,
            "key": "repair_service",
            "label": "Repair services",
            "category_url": "https://wiki.everycrt.com/wiki/Category:Repair_services",
            "ask_query": "[[Category:Repair services]]",
            "category": "Repair services",
            "description": "Repair shops and technicians serving the vintage AV community."
        },
        {
            "list_url": "https://wiki.everycrt.com/api.php?action=ask&format=json&formatversion=2&query=%5B%5BCategory%3ADocuments%5D%5D%7C%3FModification+date%7Climit%3D500%7Coffset%3D0",
            "count": 350,
            "key": "document",
            "label": "Documents",
            "category_url": "https://wiki.everycrt.com/wiki/Category:Documents",
            "ask_query": "[[Category:Documents]]",
            "category": "Documents",
            "description": "Manuals, brochures, spec sheets, service bulletins, and other reference documents."
        }
    ],
    "wiki": {
        "api_url": "https://wiki.everycrt.com/api.php",
        "name": "EveryCRT",
        "base_url": "https://wiki.everycrt.com"
    },
    "endpoints": {
        "entity_facts": {
            "url": "https://wiki.everycrt.com/api.php?action=browsebysubject&format=json&formatversion=2&subject=<PAGE_TITLE>",
            "method": "GET",
            "notes": "Returns every SMW fact (property/value pair) for a single page, including subobjects (CRT inputs, outputs, voltage ranges, etc.)."
        },
        "recent_changes": {
            "url": "https://wiki.everycrt.com/api.php?action=query&list=recentchanges&format=json&formatversion=2&rcprop=title%7Ctimestamp%7Cids%7Csizes%7Cuser&rclimit=500",
            "method": "GET",
            "notes": "Returns recent edits across all namespaces. Use rcend=<ISO timestamp> to fetch only changes since your last sync. Add &rcnamespace=N|N to filter by namespace."
        },
        "rdf_export": {
            "url": "https://wiki.everycrt.com/wiki/Special:ExportRDF?pages=<PAGE_TITLE>",
            "method": "GET",
            "notes": "Full RDF/OWL export. Use &recursive=1&pages=Category:CRT models to export an entire category subtree."
        },
        "list_entities": {
            "url": "https://wiki.everycrt.com/api.php?action=ask&format=json&formatversion=2&query=<ASK_QUERY>",
            "method": "GET",
            "notes": "Returns titles and selected printouts for entities matching the SMW ask query. Append |limit=N|offset=M for pagination (max limit 500)."
        }
    },
    "generated_at_utc": "2026-08-02T21:24:27Z"
}

See also

MediaWiki Appliance - Powered by TurnKey Linux