Convert a folder

Convert every file in a folder through the API while staying under the rate limit.

batch.mjs converts every file in a folder, a few at a time. It imports convertFile from the Node helper.

import { readdir } from "node:fs/promises";
import { join } from "node:path";
import { convertFile } from "./convt.mjs";

const [dir, to, concurrency = "4"] = process.argv.slice(2);
const workers = Number(concurrency);
// 120 requests a minute per key: poll each job every `workers` seconds so all
// jobs together poll about once a second.
const pollMs = workers * 1000;
const files = (await readdir(dir, { withFileTypes: true }))
  .filter((entry) => entry.isFile())
  .map((entry) => join(dir, entry.name));

let next = 0;
let failed = 0;
async function worker() {
  while (next < files.length) {
    const file = files[next++];
    try {
      const saved = await convertFile(file, to, { pollMs });
      console.log(`${file} -> ${saved.join(", ")}`);
    } catch (error) {
      failed++;
      console.error(`${file}: ${error.message}`);
    }
  }
}

await Promise.all(Array.from({ length: workers }, worker));
console.log(`${files.length - failed} converted, ${failed} failed`);
process.exitCode = failed ? 1 : 0;

$ node batch.mjs ./photos webp 4
photos/a.png -> a.webp
photos/b.heic -> b.webp
2 converted, 0 failed

Stay under the rate limit

Your key gets 120 requests a minute, or 2 a second. The script polls each job every N seconds when it runs N at a time, so polling stays near 1 request a second whatever the concurrency, with room left for the create, start and download calls. Raising the concurrency speeds up the folder but makes each job’s result arrive a little later.

Two files with the same name and different extensions, such as a.png and a.heic, both save as a.webp. Convert them into separate folders if that matters.

For folders on your own machine, the convt CLI does the same thing locally, with no key and no upload:

convt ./photos -r --to webp -j 4