Skip to main content

Background Tasks in PostGraphilePostGraphile 中的后台任务

In your GraphQL server typically most processing will be done in the "foreground", as part of the request-response cycle. However, since clients typically wait for this request cycle to complete before allowing the user to continue to the next task, if this processing is slow (and by slow we mean anything over a couple hundred milliseconds) it will make your application feel sluggish and can frustrate your users. Because of this desire for low-latency APIs, it's good practice to only perform essential calculations during the request-response cycle, and to "queue" additional tasks for background processing.

在 GraphQL 服务器中,大部分逻辑通常会在“前台”作为请求-响应周期的一部分同步执行。不过,客户端往往会等待请求结束后才允许用户继续下一步操作,如果处理耗时较长(哪怕只是超过几百毫秒),就会让应用显得卡顿,影响用户体验。为了保证 API 的低延迟,最佳实践是在请求-响应周期内只执行核心必要的计算,把其余耗时操作“入队”交由后台异步处理。

Examples of potential background tasks include creating PDFs, sending emails, calling an external API that may be slow, performing big calculations, and the like. Background tasks are also a good way of dealing with tasks where temporary failures are expected, for example you might not be able to send an email immediately - rather than failing the request-response cycle we can just try again (and again, and again) over the coming seconds, minutes, and hours. Another task you might want to solve with background tasks is running code periodically, detached from a user action. These are all good cases for background processing: the code runs outside of main request-response cycle, storing the result somewhere (typically on the database) and (optionally) notifying the client once it has finished.

适合放到后台执行的任务包括生成 PDF、发送邮件、调用可能较慢的外部 API、执行大规模计算等。此外,对于可能发生临时性失败的操作(比如邮件一时发送失败),后台任务也是理想的应对方案——不必直接让整个请求报错,而是可以在接下来的数秒、数分钟甚至数小时内持续重试。另一类常见场景是脱离用户请求、周期性执行的定时代码。这些场景都非常适合采用后台处理:任务在主请求-响应周期之外运行,将结果持久化(通常存入数据库),并在完成后按需通知客户端。

Task queues are systems that enable background processing for an application. They enable a way to persist this background processing tasks, notify the workers and handle errors and re-tries.

任务队列(Task Queue)正是为应用提供后台处理能力的系统,负责持久化这些后台任务、通知 worker 执行,并统一处理错误与重试逻辑。

Tasks queues and workers for Postgres适用于 Postgres 的任务队列与 Worker​

There are several existing solutions that can integrate with a PostgreSQL database or expose a Node.js-friendly API.

目前已有多种成熟方案可以与 PostgreSQL 数据库集成,或提供对 Node.js 友好的 API:

Choosing your Task Queue如何选择任务队列​

An important topic you will need to consider is how tasks are created in your queue.

选择方案时,一个需要重点考虑的问题是:任务是如何被写入队列的。

Since we generally recommend database-driven development, the recommend approach is to use Graphile Worker (or Graphile Worker + another task queue through task delegation) since you can add jobs to the queue both from inside PostgreSQL functions (and thus triggers) and from any other source.

由于我们通常倡导数据库驱动的开发方式,推荐采用 Graphile Worker(或采用 Graphile Worker 结合任务委派机制对接其他任务队列)。这样一来,你既能在 PostgreSQL 函数(以及触发器)内部直接向队列添加任务,也能从应用中的任意其他位置入队。

Other important considerations when choosing a task queue would be:

选择任务队列时,其他需要重点考量的因素还包括:

  • Complexity

    系统复杂度

  • Throughput requirements

    吞吐量需求

  • DevOps complexity

    运维(DevOps)复杂度

  • Traceability

    可追溯性

  • Privacy

    数据隐私

Graphile WorkerGraphile Worker 简介​

Graphile Worker is a natural fit for a PostGraphile stack due to its PostgreSQL-first nature. It is a simple library that runs Node.js code (or any code Node.js can delegate to) when a task is queued within the database. Thanks to PostgreSQL's LISTEN/NOTIFY pubsub features, Graphile Worker is notified when a task is queued and can fetch, execute, and complete a trivial task in 2-3ms from when it was queued; this results in your system feeling very snappy.

Graphile Worker 秉持 PostgreSQL 优先的设计理念,与 PostGraphile 技术栈天然契合。它是一个简洁的库,当数据库中有任务入队时,即可触发运行对应的 Node.js 代码(或由 Node.js 委派的任何其他代码)。借助 PostgreSQL 的 LISTEN/NOTIFY 发布订阅机制,Graphile Worker 能在任务入队后立刻收到通知,一个简单任务从入队到拉取、执行并完成仅需 2–3ms,从而让系统响应极其迅速。

Graphile Worker stores tasks into its own schema within PostgreSQL, and offers a simple SQL API to create jobs. Queueing a job might look like:

Graphile Worker 会将任务存储在 PostgreSQL 中专属的 schema 下,并提供简洁的 SQL API 来创建任务。将任务加入队列的示例如下:

select graphile_worker.add_job('hello', json_build_object('name', 'Bobby Tables'));

Tasks are created in JavaScript/TypeScript, so it fits directly into your PostGraphile stack

任务处理逻辑使用 JavaScript/TypeScript 编写,可以无缝融入现有的 PostGraphile 技术栈:

// tasks/hello.js
module.exports = async (payload, helpers) => {
const { name } = payload;
helpers.logger.info(`Hello, ${name}`);
};

Check the Graphile Worker docs for details on how to set up and use it.

有关安装与详细用法,请查阅 Graphile Worker 文档。

Limitations局限性​

Despite all the benefits, Graphile Worker stills lacks some features provided by other more mature tasks queues

尽管优势明显,Graphile Worker 目前仍缺少一些成熟任务队列所具备的功能:

  • Currently Graphile Worker doesn’t expose a GUI to monitor or manage tasks.

    目前 Graphile Worker 尚未提供用于监控或管理任务的图形界面(GUI)。

  • ??