Our previous article took an introductory look at semver from a Node.js perspective. In this article we'll be using the newly introduced caret ^ range specifier to dive deeper into how npm views semver and how we, as Node.js developers, should be thinking about how to version our packages.

Semver is fairly clear in its specification but there are details that each software community choosing to adopt semver must grapple with in order to make the ideals of semver line up with the practicalities of their platform and the norms of their community. Node.js is certainly no exception; our platform has a relatively unique approach to dependencies and our community norms have evolved towards much smaller, fine-grained packages and projects that can be constructed from complex and deep dependency trees. Our approach to semver needs to take these factors into consideration.

Version Range Sugar

The tilde ~ has been the default semver range specifier for package.json dependency definitions for two and a half years. Thanks to npm install --save, package authors have been pinning to versions within a fixed major and minor pair but a floating patch version.

For example, the second most downloaded package on npm, qs, is most commonly installed as a dependency like so:

  "dependencies": {
    "qs": "~2.2.3"
  }

Meaning that all releases from 2.2.3 up to, but not including 2.3.0 are acceptable. Even though 2.2.3 may be the current version, the author of a package depending on qs in this way is instructing npm that if new patch releases of 2.2.4 and above are available, those are acceptable. The assumption being that Nathan LaFreniere and the other maintainers of qs are not going to break any functionality depended on with a patch release and may in fact fix bugs for edge-cases users are currently unaware of.

The Need For A New Range Specifier

The caret ^ range specifier was introduced in order to also permit automatic upgrades to minor version increments of a package in order to safely inherit un-backported bug-fixes introduced in minor versions:

Theoretically this should be safe, but it's built on the assumption that package authors are strictly adhering to the semver specification regarding minor versions:

MINOR versions … add functionality in a backwards-compatible manner

Enter the caret ^ range specifier.

Not long 6 months after its introduction, the caret became the default semver save prefix in npm, so now, an npm install qs --save results in:

  "dependencies": {
    "qs": "^2.2.3"
  }

Update (16-Sep-14): Isaac has pointed out that the timing of the releases was not as close as originally suggested above. Caret was first available in npm from August 2013 and became the default save prefix 6 months later in February 2014.

Caret & Tilde: What's the Difference?

Both caret and tilde allow specifying a minimum version, and permit some flexibility as to which version will actually be installed. Neither range will be satisfied by a differing major version—the signal in semver that there are breaking changes between releases.

There are two major differences between the versions that caret and tilde capture: flexibility around minor version changes and behaviour for versions below 1.0.0 (i.e. the "magic zero" versions).

Tilde: Flexible Patch

For tilde ranges, major and minor versions must match those specified, but any patch version greater than or equal to the one specified is valid.

For example, ~1.2.3 permits versions from 1.2.3 up to, but not including, the next minor, 1.3.0.

We can demonstrate this with the semver implementation used by npm:

var semver = require('semver')

semver.toComparators('~1.2.3')
// [ [ '>=1.2.3-0', '<1.3.0-0' ] ]

Caret: Flexible Minor and Patch

For caret ranges, only major version must match. Any minor or patch version greater than or equal to the minimum is valid.

For example, a range of ~1.2.3 will only permit versions up to, but not including 1.3.0. However, the caret version, ^1.2.3 permits versions from 1.2.3 all the way up to, but not including, the next major version, 2.0.0.

semver.toComparators('^1.2.3')
// [ [ '>=1.2.3-0', '<2.0.0-0' ] ]

// compare upper limit for ~
semver.toComparators('~1.2.3')
// [ [ '>=1.2.3-0', '<1.3.0-0' ] ]

Caret: Major Zero

Given Node.js community norms around the liberal usage of major version 0, the second significant difference between tilde and caret has been relatively controversial: the way it deals with versions below 1.0.0.

While tilde has the same behaviour below 1.0.0 as it does above, caret treats a major version of 0 as a special case. A caret expands to two different ranges depending on whether you also have a minor version of 0 or not, as we'll see below:

Major and minor zero: ^0.0.z → 0.0.z

Using the caret for versions less than 0.1.0 offers no flexibility at all. Only the exact version specified will be valid.

For example, ^0.0.3 will only permit only exactly version 0.0.3.

semver.toComparators('^0.0.3')
// [ [ '=0.0.3' ] ]

semver.satisfies('0.0.4', '^0.0.3')
// false

Major zero and minor >1: ^0.y.z → 0.y.z - 0.(y+1).0

For versions greater than or equal to 0.1.0, but less than 1.0.0, the caret adopts the same behaviour as a tilde and will allow flexibility in patch versions (only).

For example, ^0.1.3 will permit all versions from 0.1.3 to the next minor, 0.2.0.

semver.toComparators('^0.1.2')
// [ [ '>=0.1.2-0', '<0.2.0-0' ] ]

// compare upper limit for ~
semver.toComparators('~0.1.2')
// [ [ '>=0.1.2-0', '<0.2.0-0' ] ]

semver.satisfies('0.1.3', '^0.1.2')
// true

semver.satisfies('0.2.0', '^0.1.3')
// false

If the changing semantics based on number of zeros seems confusing, you are not alone:

Major Zero and the Spec

The semver specification defines what has come to be known as "the escape clause" for 0.y.z versions:

Major version zero (0.y.z) is for initial development. Anything may change at any time.

In other words: normal semver rules are not in effect for major version zero, furthermore:

The public API should not be considered stable.

The whole point of semver is to make software composable and stable despite the inherent instability of individual components. Thus, it makes little sense to opt-out of full semver during the precise time when it is most useful to your consumers.

"Initial development" is very vague. What is initial development? When does initial development end? The semver FAQ gives us some clues in as to when a package should reach 1.0.0:

How do I know when to release 1.0.0? If your software is being used in production, it should probably already be 1.0.0…

While not a terrible metric, it's often interpreted as: "If your software is not being used in production, it should probably not be 1.0.0", which is not what it says.

…If you have a stable API on which users have come to depend, you should be 1.0.0.

This is the key point for package authors:

…If you're worrying a lot about backwards compatibility, you should probably already be 1.0.0.

The difference between "initial development" and "non-initial development" is probably abundantly clear to the original authors of the spec and they likely did not intend this as a comprehensive checklist, but even by these few conditions, it's clear that most Node.js packages should not be in major version zero and thus are not using semver correctly.

If your package is truly "experimental" and semver is too difficult to follow, then users are ill-advised depending on auto-upgrades. This is what's codified by the caret's conservative versioning for major version zero.

It is not clear whether "experimental" is even a useful designation for a piece of software (again, why not just version it properly from the start?) but at least the implementation used in npm now reflects the intent of the specification more closely.

Further questioning of major version zero semantics should be taken to the semver specification issue list.

1.0.0 Anxiety

Yet in reality, our community norms to date mean that there is a huge number of packages in the npm registry that never leave the safety of the major zero. ~82% of the ~94,000 packages in the npm registry have yet to hit 1.0.0.

Number of packages in the npm registry at particular major versions:

MAJOR    TOTAL PERCENT
0        77282 82.43%
1        13314 14.20%
2        2252   2.40%
3        560    0.60%
4        185    0.20%
5        67     0.07%
6        35     0.04%
7        21     0.02%
8        10     0.01%
9        24     0.03%
...
999      1      0.00%
1215     1      0.00%
2012     8      0.01%
2013     6      0.01%
2014     17     0.02%
3001     1      0.00%
4001     1      0.00%
6000     1      0.00%
9000     2      0.00%
20130416 1      0.00%
20140401 1      0.00%

Source

If the Node.js community were using semver correctly, you'd expect far more packages to reach versions >3 or higher.

Likely as a result of the long-standing behaviour of the tilde range specifier in npm, we seem to have reinterpreted the semver spec. Many package authors currently communicate breaking and non-breaking changes by condensing all version changes into the last two segments of the version, something like: 0.MAJOR.MINOR.

This "minor is for breaking changes" interpretation has remained functional while the majority of packages used the tilde–as it will not move past the current minor. However, the caret now permits minor version flexibility, preventing this interpretation from continuing to function in practice; it's now at odds with both specification and implementation.

Recommendation: Start At 1.0.0

The manner in which the caret changes semantics of npm package versioning has been so thoroughly distasteful for some developers they are simply avoiding zero majors entirely:

Pushing developers through the imaginary 1.0.0 barrier has the nice side-effect of getting developers to start using semver correctly. i.e. bump the major whenever you break the API and ignore any arbitrary, sentimental values you're assigning to major version numbers–they're just numbers.

This has also prompted a change to allow default version for new packages created with npm init to be configurable. Subsequently, npm's default setting for package versions has been changed from 0.0.0 to 1.0.0 as of npm version 1.4.22, which means that the npm bundled with Node.js version 0.10.31 and later have this change.

The caret does allow much more flexibility than tilde, which giving some people cause for panic. The caret requires consumers must put more trust in authors to follow the semver specification, but the Node.js community hasn't been particularly good at following the semver specification at all, and this is primarily due to that unwillingness to break through the 1.0.0 barrier.

The Caret and Node.js 0.8 Fiasco

Since the Node.js 0.6.3 release in 2011, Node.js has been bundled with the latest version of npm as at time of release. This helps bootstrap the Node.js experience, and is a good thing except when users can no longer use their bundled npm to install dependencies due to incompatible changes in the npm client and/or registry:

• Node.js 0.8.23 and above are bundled with npm 1.2.30
• Caret support is first available in npm 1.3.7, released in early August, 2013
• The first version of Node.js to bundle npm with any degree of support for the caret was Node.js 0.10.16 in late August, 2013
• The default save prefix is set to caret in npm 1.4.3 in early February, 2014
• The first version of Node.js to bundle npm with caret as the default save prefix is Node.js 0.10.26, released in late February, 2014

After the release of Node.js 0.10.26, many packages using the shiny new default caret operator start appearing in the registry. Everyone on Node.js 0.10.15 and below using their bundled npm start receiving unprovoked "No compatible version found" errors during installation.

Users on early versions of 0.10 are told to upgrade to get a newer version of npm with caret support, which is easy enough, but the big problem is that as of writing, there is no version of Node.js 0.8 with a caret-compatible npm, thus the default npm bundled with Node.js on 0.8 is simply broken.

Despite 0.10 being the current stable version of Node.js for nearly 18 months, there are still users running Node.js 0.8 for various reasons, and their bundled npm was functioning just fine until the deluge of carets started appearing in the registry.

The advice for Node.js 0.8 users is to simply update npm using npm:

npm install -g npm

Caret Is the New Norm

As of npm version 1.4.3, the caret semver range specifier is the new default prefix for writing versions into package.json using npm install --save, npm install --save-dev, etc.

If you would prefer to opt-out of the new caret default, you can configure your default save-prefix to back to tilde:

npm config set save-prefix '~'

Hopefully you are now equipped to make a more informed decision about how you want to specify your version ranges in your package.json files. But above all, go forth and use semver properly, for the sake of the entire Node.js community!

If you're looking for even more information on npm version ranges, listen to NodeUp #70, an npm client show where we discuss caret and some of the issues surrounding it.