@blueforge-studio/image-to-3d

Provider-agnostic image → 3D client. One typed interface for GAS (vault7-image-to-3d job) and bevy-game-editor (ai.image_to_3d MCP tool); GPU/provider details stay behind ImageTo3DProvider.

• Spec: game-asset-platform/docs/superpowers/specs/2026-10-12-image-to-3d-trellis2-design.md (PR #169)
• Browser-safe: no Node built-ins (the editor webview imports it).
• No credentials inside: providers receive keys/endpoints from the caller.

Usage

import { generateModel, MockProvider } from "@blueforge-studio/image-to-3d";

const r = await generateModel(provider, { imageUrl: "https://…/a.png", resolution: 1024 }, { licenseUse: "commercial" });
// r.glb (validated), r.meta, r.report.warnings, r.provider, r.jobId

licenseUse is required: a provider whose stack is non-commercial (or review-required) is refused for commercial requests. validateGlb enforces the output contract (glTF 2.0 binary, embedded textures, ≤20 MB, ≤100k faces; Y-up/metres/origin reported as warnings). MockProvider + makeFixtureGlb let consumers test without a GPU.

Quality presets and cost

Three request fields trade cost for fidelity: resolution (how hard the model works; the main GPU cost), targetFaces (mesh weight) and textureSize (file size). Presets bundle them:

import { buildRequest, estimate, refineRequest, generateModel } from "@blueforge-studio/image-to-3d";

const req = buildRequest({ imageUrl, preset: "game", textureSize: 1024 }); // explicit fields beat the preset
const e = estimate(req);               // { seconds, usd, confidence: "measured" | "extrapolated" | "unmeasured", note, ... }
const draft = buildRequest({ imageUrl, preset: "draft", seed: 7 });
const final = refineRequest(draft, "standard"); // same image + seed, higher quality

Precedence: explicit fields > preset > schema defaults. undefined overrides are ignored.

What the numbers mean (be careful). estimate uses real A40 runs of the shipped worker (2026-10-13; RunPod secure cloud, $0.59/h; one image, one seed, so a sample, not a benchmark; the 2026-10-12 spike saw 1024 generations range 30-159 s across inputs). Warm generation: 18.0 s at 512, 45.5 s at 1024 (independent of face count and texture size). Warm bake+export by settings actually run: 8.6 s (512 / 20k / 1024), 16.8 s (512 / 100k / 2048), 21.3 s (1024 / 30k / 2048), 26.7 s (1024 / 100k / 2048). So standard is about 72 s (about $0.012), draft about 27 s, game about 67 s. measured means those exact settings were run; extrapolated reuses the slowest measured bake at the same resolution with at least those faces and texture size (an upper bound); settings above everything measured, and 1536 (hero), return null rather than an invented number. One-off costs are separate fields: coldStartSeconds 143 (weights download + load; 132 s and 143 s measured) and firstRunWarmupSeconds 76 (the first generation after a load took 94 s against 18 s warm; the first 1024 run after 512 added about 31 s more). Peak VRAM was 3.45 GB at 512 and 6.44 GB at 1024. Quality: the standard output was a faithful boot (eyelets, laces, tread, printed pull tab); the draft was recognisably the same boot but visibly coarser. The draft and game face counts remain proposals, not tuned values.

Draft-then-refine is unverified. refineRequest keeps the image and seed on the hypothesis that the same seed gives a similar shape at a higher resolution (the 1024 pipeline is a cascade from 512). Check it on real hardware before relying on it.

MockProvider records every request in provider.requests and echoes the request's seed and resolution in meta; it cannot honour targetFaces/textureSize (the fixture mesh does not change). It is for tests only.

RunPod provider (trellis2-runpod)

RunPodTrellis2Provider drives the trellis2-worker image behind a RunPod serverless endpoint (POST /v2/{endpoint}/run, GET .../status/{job}, POST .../cancel/{job}). The worker uploads the GLB to a presigned PUT URL you supply and the provider reads it back from the matching GET URL, so you bring the object store (B2/S3).

import { RunPodTrellis2Provider, buildRequest, generateModel } from "@blueforge-studio/image-to-3d";

const provider = new RunPodTrellis2Provider({
  endpointId, apiKey,                         // injected by you; this package reads no env
  outputTargets: async (req) => presign(req), // -> { glbPutUrl, glbGetUrl, metaPutUrl? } (https, presigned)
  licenseUse: "non-commercial",               // default is "review-required", which generateModel refuses
  policy: { executionTimeoutMs: 600_000 },
});
const r = await generateModel(provider, buildRequest({ imageUrl, preset: "standard" }), { licenseUse: "non-commercial" });

• Licence: licenseUse defaults to review-required because the DINOv3 licence review has not been done (the operator skipped it for technical tests). Until it is, generateModel refuses the provider unless you state a licence explicitly, and outputs are not for commercial use.
• Errors: the worker's own codes (MATTING_FAILED, OOM, EXPORT_FAILED, ...) pass through; RunPod FAILED -> PROVIDER_ERROR, TIMED_OUT -> TIMEOUT (retryable), CANCELLED -> CANCELLED. Presigned URLs and the API key never appear in error messages. A few transient status-poll failures (429, 5xx, network) are tolerated before a job is failed.
• Endpoint settings that the measured runs support: container disk 100 GB (image is 34.7 GB uncompressed), a GPU with >= 24 GB (an RTX 4090 and an A40 were measured; peak VRAM 6.4 GB at 1024), HF_TOKEN as a secret env var, scale to zero, an execution timeout of at least 10 minutes (cold start is about 2.5 min of weight load plus a first-run warm-up of 50-75 s before a generation of about 30-50 s), and a registry credential for registry-ext.blueforge.studio.
• State: the provider remembers each job's download URL in memory, so a job cannot be resumed by a different process.
• Status: not yet exercised against a live endpoint. The 32 provider tests use a faked RunPod and object store (mutation-checked: removing the URL scrubbing, flipping the licence default, leaking the key, or removing the transient-error limit each makes a test fail). The paths and status values were checked against docs.runpod.io; the first live job should run on a throwaway endpoint.

Status

Skeleton: schemas, provider interface, generateModel, GLB validator, quality presets and cost estimate, mock provider. The trellis2-runpod provider exists and is unit-tested but has not run against a live endpoint; hosted-provider adapters are not written. The worker behind it works end to end (see the trellis2-worker README); licensing: nvdiffrast is replaced, and the DINOv3 licence review is pending (spec §2.1).

Develop

npm ci && npm test && npm run typecheck && npm run build