Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pub seems to get constraints of semantic versions wrong #20404

Closed
mkustermann opened this issue Aug 7, 2014 · 6 comments
Closed

Pub seems to get constraints of semantic versions wrong #20404

mkustermann opened this issue Aug 7, 2014 · 6 comments
Labels
closed-as-intended Closed as the reported issue is expected behavior

Comments

@mkustermann
Copy link
Member

A package "foobar" with version "0.0.3-dev" satisfies the following version constraint:

dependencies:
  foobar: '>=0.0.3-dev <0.0.3'

according to semantic versioning, see the specification here: http://semver.org/

But pub seems to complain that there is no suitable version for "foobar" -- could be because the constraints are parsed incorrectly or because the ordering implementation is wrong?

Error message:
$ pub get
Resolving dependencies...
Package foobar has no versions that match >=0.0.3-dev <0.0.3 derived from:

  • mypackage 0.1.0-dev depends on version >=0.0.3-dev <0.0.3
@DartBot
Copy link

DartBot commented Aug 7, 2014

This comment was originally written by @zoechi


Isn't this what was changed with this issue https://code.google.com/p/dart/issues/detail?id=20302 ?

@mkustermann
Copy link
Member Author

Seems so. But issue #20302 has been marked as fixed, and the issue I see remains on bleeding_edge. I also don't understand why there is a discussion about this:

The semantic versioning standard defines the ordering of versions. Therefore the <, <=, >= and > operators must be implemented according to the spec -- and they're not.

So let's keep this bug open until we follow the specification (we advertise everywhere that we follow semver, so we should do that).

@munificent
Copy link
Member

This is working as intended.

Therefore the <, <=, >= and > operators must be implemented according to the spec -- and they're not.

Pub fully implements semver (semver 2.0.0-rc.1 specifically, because 2.0.0 final is dumb). If you compare two version numbers, you will get the behavior you describe. But, short of importing code from pub itself, there is no user-facing way to actually compare two version numbers.

Instead, the behavior you see is testing a version number against a constraint object. Semver doesn't describe or prescribe version constraints, either their syntax or semantics. I could define a version constraint that only permitted odd-numbered versions and semver would be fine with it.

I fully admit that the syntax for version constraints, using comparison operators, doesn't make this obvious. If we had a different syntax for constraints (like "~1.2.3" or "1.2.x" or "upto 1.2.3"), there'd be no confusion here even though the semantics would be exactly the same.

In most common cases, a "<2.0.0" constraint does work as if its semantics were "admits any version where '(version) < 2.0.0' is true". (That's why the syntax is useful and intuitive.) In the presence of development releases, though, the behavior is different. The difference here is important because it follows the semantics of what the versions represent.

If you you have a package that works with all of the 1.x.y releases of foo, it clearly won't be expected to work with foo 2.0.0. So the natural, correct constraint is <2.0.0. But it's also certainly the case that if your package doesn't work with foo 2.0.0, it's not likely to work with random experimental development releases of 2.0.0!. Given that, if a <2.0.0 constraint allowed 2.0.0-dev, that wouldn't do anything good for you.

One way to look at this is that a "<2.0.0" range is really just a concise way to write "<=1.INT_MAX.INT_MAX".

This change brings pub inline with npm and RubyGems/bundler, which both also exclude pre-release versions in a "<" constraint (unless the constraint itself is a pre-release version). See:

http://bundler.io/v1.3/gemfile.html
https://www.npmjs.org/doc/misc/semver.html


Added AsDesigned label.

@mkustermann
Copy link
Member Author

semver 2.0.0-rc.1 specifically, because 2.0.0 final is dumb.

Could you explain the difference here?
Do we have documentation that we don't follow the normal semver spec but rather a previous release candidate of it?

Semver doesn't describe or prescribe version constraints, either their syntax or semantics.

Then please explain to me why they describe an ordering of versions in the first place? I think they define the ordering so you can see if a version is below an upper bound or above a lower bound. This is indicated by the usual operators (<, <=, >, >=). Overloading these operators with a different behavior causes high confusion.

Is there then a specification of what semantics the pub "constraint objects" have? [which includes a description on pre-release versions and whether/if they can be used for published packages]

The difference here is important because it follows the semantics of what the versions represent.

I am fully aware of the problem with pre releases. But pub's behavior is just inconsistent:

You say pub's "constraint objects" have nothing to do with the order of semantic versions. You want to have the right behavior for users. Which is -- I agree here -- that you compare only stable versions (maybe including build metadata). Even the semver spec says:

"A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version"

But pub on the other hand says:

"0.0.4-dev" does not satisfy the constraint 'foobar: >=0.0.3 <0.0.4'
"0.0.4-dev" does satisfy the constraint 'foobar: >=0.0.3 <0.0.5'

This is inconsistent behavior! If I use a feature in foobar which works in 0.0.3 and in 0.0.4 I would use the second constraint (i.e. I widen the foobar constraint). But following the users expected behavior (and your logic) I do not want to get broken in-between pre-release versions of "foobar" which might have broken the feature I use. But with the current behavior I could get broken pre-releases.

=> The conclusion is:
Either allow pre-releases in ">=a.b.c <x.y.z" constraints or you don't allow them (user would prefer the second one). But please be consistent here. You could also make an opt-in feature for pre-releases.

I'd strongly encourage you to make this consistent!

@munificent
Copy link
Member

Could you explain the difference here?

Semver 2.0.0 final specifies that there is no ordering between versions who only differ by build number. rc.1 says that they're ordered the same way pre-releases are. The motivation behind that change is that they want build suffixes to work like metadata or comments such that they don't affect any visible behavior.

However, that leaves any real-world package manager in a bad position. Let's say there are two packages on pub.dartlang.org: foo 1.0.0+a and foo 1.0.0+b. A user depends on "foo". Which version should pub give them. With semver 2.0.0, they just don't have an answer. Pick one randomly? Base it on the day of the week? Who knows!

rc.1 specifies that (you'll get 1.0.0+b). This is particularly important for pub because our guidelines for versioning packages maintained by the Dart team use build suffixes in an ordered way.

Note that when pub was created, semver 2.0.0-rc.1 was the latest version of semver. 2.0.0 final didn't come out until after we'd shipped pub.

Do we have documentation that we don't follow the normal semver spec but rather a previous release candidate of it?

We've mentioned it in a few places, but, no, we don't. Filed a bug: https://code.google.com/p/dart/issues/detail?id=20422

Note that the subtleties here don't affect most users.

Then please explain to me why they describe an ordering of versions in the first place? I think they define the ordering so you can see if a version is below an upper bound or above a lower bound.

We use ordering for more than just bounds checking. It's also used to pick the best from a set of valid versions.

Is there then a specification of what semantics the pub "constraint objects" have? [which includes a description on pre-release versions and whether/if they can be used for published packages]

They're documented here: https://www.dartlang.org/tools/pub/dependencies.html#version-constraints

Like you note, they don't cover the recent change for pre-release versions.

I am fully aware of the problem with pre releases. But pub's behavior is just inconsistent:

You say pub's "constraint objects" have nothing to do with the order of semantic versions.

I didn't say they have nothing to do with it. In fact, I clearly said in most cases the behavior is the same as ordering.

You want to have the right behavior for users. Which is -- I agree here -- that you compare only stable versions (maybe including build metadata). Even the semver spec says:

But pub on the other hand says:

"0.0.4-dev" does not satisfy the constraint 'foobar: >=0.0.3 <0.0.4'
"0.0.4-dev" does satisfy the constraint 'foobar: >=0.0.3 <0.0.5'

This is inconsistent behavior! If I use a feature in foobar which works in 0.0.3 and in 0.0.4 I would use the second constraint (i.e. I widen the foobar constraint). But following the users expected behavior (and your logic) I do not want to get broken in-between pre-release versions of "foobar" which might have broken the feature I use. But with the current behavior I could get broken pre-releases.

Actually, pub has another bit of subtlety here to handle this case. When picking the "best" version to match a constraint, it prefers any stable version over any unstable version. Let's say these versions of foobar exist:

foobar 0.0.2
foobar 0.0.3
foobar 0.0.3-dev
foobar 0.0.4
foobar 0.0.4-dev
foobar 0.0.5
foobar 0.0.5-dev

You place an (unusual!) "foobar: >=0.0.3 <0.0.5" constraint on it. That filters the list down to:

foobar 0.0.3
foobar 0.0.3-dev
foobar 0.0.4
foobar 0.0.4-dev

Of those four, pub's version solver then orders them in terms of priority. That decides which versions it tries first before backtracking and picking a different version on a constraint failure. Right now, the above list is ordered strictly according to semver's ordering. But pub's solver prioritizes them (from worst to best) like this:

foobar 0.0.3-dev
foobar 0.0.4-dev
foobar 0.0.3
foobar 0.0.4

So it will try 0.0.4, then 0.0.3, and only then will it start trying unstable pre-releases. What this means in practice is that the only way to get a pre-release version is to have a constraint that permits no stable versions.

Either allow pre-releases in ">=a.b.c <x.y.z" constraints or you don't allow them (user would prefer the second one). But please be consistent here. You could also make an opt-in feature for pre-releases.

Yes, I agree with your conclusion here. "Opt-in" is exactly how we described the way de-prioritizing pre-releases works. Most users won't see pre-release versions at all. The only way to get one is to explicitly choose a narrow enough constraint that it permits no stable versions.

@DartBot
Copy link

DartBot commented Jun 5, 2015

This issue has been moved to dart-lang/pub#1084.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
closed-as-intended Closed as the reported issue is expected behavior
Projects
None yet
Development

No branches or pull requests

4 participants