API Documentation

GYD.AI API

Three primitives — Fetch, Map, Crawl — powered by the same stealth engine. Get an API key, send a request, get clean data.

Quickstart

Get an API key from /api-keys, then run:

curl -X POST http://localhost/api/v1/fetch \
  -H "x-api-key: $GYD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com"]}'

Authentication

Send your API key as the x-api-key header on every request. Keys are scoped per workspace and revocable from the dashboard.

Base URL: http://localhost/api/v1

Endpoints

Fetch — render a single page

Returns the rendered HTML and clean Markdown for one or more URLs.

curl -X POST http://localhost/api/v1/fetch \
  -H "x-api-key: $GYD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com"],
    "proxy": { "type": "datacenter" }
  }'

Map — discover a site's URL graph

Walks sitemaps and robots.txt to return every reachable URL on a domain.

curl -X POST http://localhost/api/v1/map \
  -H "x-api-key: $GYD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com"],
    "proxy": { "type": "residential" }
  }'

Crawl — fetch a whole section of a site

Discovers URLs from the seed page and fetches each one through the same stealth pipeline as /fetch. Returns clean Markdown per page plus a downloadable bundle.

Note: max_depth is currently capped at 1. Multi-depth traversal is on the roadmap.

curl -X POST http://localhost/api/v1/crawl \
  -H "x-api-key: $GYD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com"],
    "max_depth": 1,
    "max_pages": 100,
    "proxy": { "type": "datacenter" }
  }'

Check job status

All three endpoints return a job_id that you poll until status is completed or failed.

curl http://localhost/api/v1/fetch/<job_id> \
  -H "x-api-key: $GYD_API_KEY"

Response shape

Every POST returns a queued job descriptor. Single-URL requests get convenience fields job_id and poll_url; batch requests use the plural job_ids + poll_urls arrays.

{
  "service": "fetch",
  "status": "queued",
  "count": 1,
  "accepted_count": 1,
  "rejected_count": 0,
  "duplicates_removed": 0,
  "rejected": [],
  "job_ids": ["batch_abc123"],
  "poll_urls": ["/api/v1/fetch/batch_abc123"],
  "job_id": "batch_abc123",
  "poll_url": "/api/v1/fetch/batch_abc123"
}

Idempotency

Safe-retry any POST by sending an Idempotency-Key header. Repeated calls with the same key return the original response and don't double-charge or duplicate work. Keys are scoped per API key and expire after 24 hours.

curl -X POST http://localhost/api/v1/fetch \
  -H "x-api-key: $GYD_API_KEY" \
  -H "Idempotency-Key: my-unique-request-id" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com"]}'

Webhooks

Skip polling — pass webhook_url in your request body and we'll POST the completion payload to your endpoint. Add webhook_secret to receive an x-gyd-signature: sha256=<hex> HMAC header so you can verify the call came from us.

curl -X POST http://localhost/api/v1/fetch \
  -H "x-api-key: $GYD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "urls": ["https://example.com"],
    "webhook_url": "https://your-server.com/gyd-hook",
    "webhook_secret": "your-shared-hmac-secret"
  }'

Retries with exponential backoff: 2s → 8s → 32s. Webhook is fire-and-forget — your endpoint failures don't fail the original job.

Verify the signature on your server

Always verify x-gyd-signature before trusting the payload — it confirms the request came from us and wasn't replayed or tampered with. Use a constant-time comparison (crypto.timingSafeEqual / hmac.compare_digest) to avoid timing attacks.

Node.js (Express)

import crypto from "crypto";

// Express handler. Use raw-body so the HMAC matches what we signed.
app.post("/gyd-hook", express.raw({ type: "application/json" }), (req, res) => {
  const signature = req.header("x-gyd-signature") || "";
  const expected = "sha256=" + crypto
    .createHmac("sha256", process.env.GYD_WEBHOOK_SECRET)
    .update(req.body)             // raw Buffer
    .digest("hex");

  if (!crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected))) {
    return res.status(401).send("invalid signature");
  }

  const payload = JSON.parse(req.body.toString("utf8"));
  // ... handle payload (job_id, service, status, outputs, ...)
  res.json({ ok: true });
});

Python (Flask)

import hmac, hashlib, os
from flask import Flask, request, abort

app = Flask(__name__)

@app.post("/gyd-hook")
def gyd_hook():
    signature = request.headers.get("x-gyd-signature", "")
    expected = "sha256=" + hmac.new(
        os.environ["GYD_WEBHOOK_SECRET"].encode(),
        request.get_data(),       # raw bytes
        hashlib.sha256,
    ).hexdigest()
    if not hmac.compare_digest(signature, expected):
        abort(401)
    payload = request.get_json()
    # ... handle payload
    return {"ok": True}

Errors

Errors return a JSON body with error and an HTTP status code:

StatusMeaning
400Invalid request body or unsupported parameter.
401Missing x-api-key or unauthenticated dashboard session.
402Out of credits — top up from the dashboard.
403Invalid API key, or key lacks the scope for this endpoint (fetch / map / crawl).
404Job ID not found or expired.
429Rate limit exceeded — retry with exponential backoff.
5xxServer-side error — safe to retry.

OpenAPI Spec

The full API contract is available as an OpenAPI 3.0 spec. Import it into Postman, Insomnia, or your editor of choice for autocomplete and request testing.

Download openapi.v1.yaml

Have questions or want SDK support for your language? Get in touch.