Share

Exciting news this week! One of the latest features in Node.js core is the addition of support for requiring synchronous ECMAScript Modules (ESM) graphs. This enhancement promises to simplify the transition for package authors and users alike, as the ecosystem gradually shifts towards ESM. This long-awaited feature, enabled via the --experimental-require-module flag, marks a pivotal moment in Node.js development, addressing a persistent pain point for developers.

In this blog post, we explore deeper into the journey leading to this milestone, exploring the technical and cultural intricacies that shaped its evolution.

Understanding the Update

The pull request aims to address a long-standing pain point for Node.js users: the inability to require ESM modules synchronously. While Node.js has supported ESM for some time, requiring these modules has been restricted to asynchronous operations using import(). However, with the introduction of this new feature, developers can now use require() for synchronous loading of ESM modules.

How It Works

The implementation of this feature relies on the --experimental-require-module flag. When enabled, Node.js will allow synchronous loading of ESM modules via require(). However, there are certain conditions that must be met for this to work:

1. The ESM module must be explicitly marked as such, either through a "type": "module" field in the closest package.json or by using a .mjs extension.
2. The module must be fully synchronous, meaning it contains no top-level await expressions.

If these conditions are met, require() will load the module as an ESM and return the module namespace object synchronously. This behavior mirrors dynamic import() but operates synchronously.

Background and Motivation

The decision to introduce this feature stems from ongoing discussions within the Node.js community. Previous attempts to support synchronous loading of ESM modules faced challenges, particularly in handling top-level await expressions. However, with the current implementation, the focus remains on simplicity and compatibility.

The motivation behind this enhancement is clear: to ease the transition for package authors and users as the ecosystem embraces ESM. By allowing synchronous loading of ESM modules, package authors can migrate their codebases without worrying about breaking changes for users. This not only streamlines the transition process but also reduces concerns about node_modules bloat and identity issues due to duplication.

The Agony of ERR_REQUIRE_ESM

For years, Node.js developers grappled with the frustration of ERR_REQUIRE_ESM. While importing CommonJS (CJS) modules was straightforward, requiring ESM modules remained elusive. This discrepancy led to confusion and wasted hours, particularly for those unaware of the underlying complexities. Package authors faced dilemmas, forced to choose between maintaining compatibility with both CJS and ESM users or risking breaking changes. Meanwhile, the prevailing belief that "ESM is async" perpetuated a myth, obscuring the true nature of ESM syntax.

const open = require('open');

open('https://nodesource.com');

We are using the popular open module for opening files, and we have in our project the following dependency:

{
  "dependencies": {
  "open": "^8.4.2"
  }
}

Everything works correctly and the script opens a browser pointing to https://nodesource.com.

If we want to update the dependency to the latest stable one, we will have to execute:

$npm i open@latest

and we see the dependencies as:

{
  "dependencies": {
  "open": "^10.1.0"
  }
}

Now when running the script... boom! We encounter the following error:

$ node index.js 
/tmp/open/index.js:1
const open = require('open');
     ^
Error [ERR_REQUIRE_ESM]: require() of ES Module /tmp/open/node_modules/open/index.js from /tmp/open/index.js not supported.
Instead change the require of /tmp/open/node_modules/open/index.js in /tmp/open/index.js to a dynamic import() which is available in all CommonJS modules.
at Object.<anonymous> (/tmp/open/index.js:1:14) {
  code: 'ERR_REQUIRE_ESM'
}

Node.js v20.10.0

Discovering Synchronous ESM Potential

A pivotal realization occurred when exploring the inner workings of ESM syntax. Contrary to popular belief, ESM was not inherently asynchronous but rather conditionally so, triggered only by the presence of top-level await expressions. This insight laid the groundwork for reconsidering synchronous require(esm), a concept previously explored in 2019 but overshadowed by technical challenges and heated debates.

A New Approach Emerges

Despite the complexities, a renewed effort to implement synchronous require(esm) gained momentum. A fresh perspective led to a simplified approach, focusing solely on supporting synchronous ESM graphs. This pragmatic approach garnered support, paving the way for the feature's inclusion in Node.js core.

To avoid the error shown before, you can do the following using the open module which only supports ESM:

const {  default: open } = require('open');

open('https://nodesource.com');

And then run:

node --experimental-require-module index.js

That’s it! No more errors.

Looking Ahead

While the current implementation addresses the majority of use cases for synchronous ESM loading, there are still areas for improvement. Certain feature interactions, such as with experimental flags like --experimental-detect-module or --experimental-wasm-modules, may need further consideration. Additionally, edge cases involving cyclic dependencies might require additional handling.

However, as with any experimental feature, the goal is to iterate and improve over time. By introducing this capability as an experimental feature, Node.js can gather feedback from users and continue refining the implementation. As the JavaScript ecosystem evolves, Node.js remains committed to providing a robust platform for developers.

In conclusion, the addition of support for synchronous ESM graphs represents a significant step forward for Node.js. It not only simplifies the transition to ESM for package authors but also enhances the overall developer experience. With this new capability, Node.js continues to adapt and innovate, ensuring it remains a leading platform for server-side JavaScript development.

References: require(esm) in Node.js - Joyee Cheung's Blog