Added Area-Pub, Triaged labels.

When we were first designing the package system, we intentionally decided to limit the number of different ways a dependency could be declared to encourage both conceptual clarity and straightforward implementation. I think that principle applies here: there isn't enough benefit from not having some dependencies downloaded under some circumstances to justify the complexity cost.

Added NotPlanned label.

This comment was originally written by @seaneagan

Example of why this would be useful:

https://github.com/seaneagan/which.dart/issues/3

This comment was originally written by @seaneagan

Version lock is probably the bigger concern here, not downloading extra dependencies. This bit me again, causing me to have to put my executable and library in separate packages:

http://github.com/seaneagan/den
http://github.com/seaneagan/den_api

And it just came up again here:

https://chromiumcodereview.appspot.com/1022963002/

cc @munificent.

This comment was originally written by @seaneagan

I think this would get much easier to implement once we have the package spec:

https://github.com/lrhn/dep-pkgspec

Then all the deps can still be downloaded into the same place, but pub run (when running a script from another package's 'bin') and pub global run just use a different package spec which includes bin_dependencies.

I'm going to re-open this because I'm increasingly hearing this is a problem and I don't want people to solve it by splitting their packages into pure lib / command line halves. Totally agree that package-spec will help here.

Added Triaged label.

Yes please! This is something we're tangling significantly with in analyzer-land.

A package's executables are part of its public API; it's valid for one package to depend on another because it wants to run that dependency's executable. If that executable's dependencies are distinct from its normal dependencies, that use-case breaks.

This is analogous to a package exposing two classes, one of which has a dependency that the other does not. If a downstream package only uses one class, sure, it would be nice for it not to use the other class's dependency, but that's not how our dependency system works, and for good reason: if every package has to spell out exactly which dependencies are used by which pieces of its API, and which pieces of the APIs of its dependencies it uses, the whole system becomes untenably complex.

And just like the multiple-classes case, if the dependencies or versioning story of two pieces of an API are different enough that they're causing problems, I don't think there's anything wrong with splitting them apart. We're planning on doing this with unittest, for example: there will be one "frontend" package for defining tests and a separate "backend" package for manipulating the infrastructure behind the definition, separated out so they can be versioned separately.

This comment was originally written by @seaneagan

A package's executables are part of its public API; it's valid for one package to depend on another because it wants to run that dependency's executable.

A package shouldn't depend on a package solely for its executable, that's what pub global activate is for.

If that executable's dependencies are distinct from its normal dependencies, that use-case breaks.

I think "breaks" is too strong. As mentioned in the original post, yes, it's harder for pub run, but you didn't address my proposal to build two separate package graphs, one which excludes bin_dependencies, and one which includes them.

This is analogous to a package exposing two classes, one of which has a dependency that the other does not. If a downstream package only uses one class, sure, it would be nice for it not to use the other class's dependency, but that's not how our dependency system works, and for good reason: if every package has to spell out exactly which dependencies are used by which pieces of its API, and which pieces of the APIs of its dependencies it uses, the whole system becomes untenably complex.

I don't think anyone has or would suggest such a complex partitioning of dependencies. A better analogy is with dev_dependencies. Those are useful because for example packages want to co-locate their tests, build scripts, etc. with their public API, using the standard pub package layout conventions (/lib, /test, /tool), but they don't want dependers to inherit e.g. unittest, grinder, etc. dependencies. Similarly, packages want to co-locate one cohesive public API including an executable and a dart API, using the standard pub package layout conventions (/lib, /bin) without having dependers inherit e.g. args, prompt, and ansicolor.

And just like the multiple-classes case, if the dependencies or versioning story of two pieces of an API are different enough that they're causing problems, I don't think there's anything wrong with splitting them apart. We're planning on doing this with unittest, for example: there will be one "frontend" package for defining tests and a separate "backend" package for manipulating the infrastructure behind the definition, separated out so they can be versioned separately.

That sounds like a good solution, but to a problem that's specific to the unittest package. Exposing executables however is a problem shared by a large percentage of packages. It's already partly solved by the /bin convention, pub run, etc. But partitioning bin dependencies is still a missing piece.

A package's executables are part of its public API; it's valid for one package to depend on another because it wants to run that dependency's executable.

A package shouldn't depend on a package solely for its executable, that's what pub global activate is for.

Not true. It's helpful to depend on packages that you only use executables from because it ensures everyone hacking on your package can easily run the right version of those executables. You can just tell people:

1. Clone the package.
2. Run pub get.
3. Run pub run some_dep:some_exe

However, I'm not opposed to something like bin_dependencies, though it can be a tricky to implement correctly.

A package shouldn't depend on a package solely for its executable, that's what pub global activate is for.

This is incorrect. "pub global activate" is for end-users to install executables onto their systems for their direct use. If one package needs to use another's executable, it absolutely should depend on that package; making its users run "pub global activate" is an annoyingly manual installation step that interferes with the user's system state and doesn't respect version constraints.

I want to emphasize again that we consider a package's executables to be part of its API, and this model is not likely to change. Even if we do decide to add some notion of bin dependencies, it will be in a way that preserves works with this model, rather than against it.

you didn't address my proposal to build two separate package graphs, one which
excludes bin_dependencies, and one which includes them.

How would this help? The bin-inclusive package graph would be identical to package graphs today, and if it experiences version lock, installation would still fail.

This comment was originally written by @zoechi

I'm fighting a lot with package dependencies lately and every single dependency adds more headaches. Here its with examples https://bitbucket.org/andersmholmgren/shelf_auth/issue/6/please-make-it-compatible-with-shelf-060 (see end of the discussion)
I think the dependencies should be at least be split for use as dependency in another package and everything else (test, bin, example, tool)

I think the dependencies should be at least be split for use as dependency in another package and everything else (test, bin, example, tool)

This is exactly what dev_dependencies do, except that bin shouldn't be in that list. But for things in test, example, and tool, by all means do use dev_dependencies for those.

This comment was originally written by @seaneagan

It's helpful to depend on packages that you only use executables from because it ensures everyone hacking on your package can easily run the right version of those executables.

I agree that that's a nice to have, but if folks are experiencing version lock they won't even get past the install, and thus won't be able to run the executables at all. This is not a problem with pub global activate, which leads me to my proposal!

Instead of trying to jam in the dependencies of all the binaries of all your transitive dependencies into a single dependency graph, similar to pub global activate, each of your (immediate) dependencies binaries deserve their own dependency graph. Consider that a large percentage of the time, a depender doesn't want or need to call the binaries of its dependencies, so it should only have to bear the cost in cases where it does. The cost I propose is that in the workflow you mentioned:

You can just tell people:

1. Clone the package.
2. Run pub get.
3. Run pub run some_dep:some_exe

We add a step between 2. and 3., which is to run a new command:

  pub activate some_dep

This is just like:

  pub global activate some_dep <some version or path or git repo>

except that you never need to specify <some version or path or git repo> since that is already done for you by the pubspec. It only uses the dependencies of some_dep, including its bin_dependencies, and thus some_dep is allowed to make different choices about the dependencies of its binaries than the local package that depends on it, and from binaries of any other dependencies. Also, bin_dependencies of any transitive dependencies never even come into play. This also increases the reliability and testability of the binaries, since the dependency graphs are the same regardless of whether they are activated globally or in the context of a local package.

If there is desire to easily activate all the binaries of one's immediate dependencies that could just be a pub activate command (without an argument), but I'm not sure if that's necessary.

The lock files of the pub activated binaries could be stored in a sub-directory of the .pub directory. There shouldn't be any need to check them in, since checking in the main pubspec.lock would already pin you to a specific version of the binary. Presumably the "package spec" files for each binary (once they are a thing) would co-located there.

Then when pub get or pub upgrade are later run again, there would be two options:

1. Mark the pub activated packages as dirty, so that later when they are again pub run, an error message tells the user that they need to re-pub activate the dependency.
2. Automatically re-pub activate any already activated packages for the user.

I personally like 2. a lot.

And pub activate can be added in a backwards compatible way. When pub run some_dep:some_exe is executed, it should check if some_dep has been pub_activated, and if not, it falls back to the existing behavior of using the local package's dependency graph. But this would be deprecated, and in dart 2.0 running pub activate before pub run will be required.

What do you all think?

fyi, proposal doc: https://docs.google.com/document/d/1lvbzyivCejhTuHLG4WI3YVThbQqJhPUrwajwTC-sXWU/edit

looks great to me

This comment was originally written by @seaneagan

Another example:

https://github.com/dart-lang/analyzer_cli

This comment was originally written by @seaneagan

Another package:args conflict in issue #23349.

This issue has been moved to dart-lang/pub#1231.