Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easy for pub packages to ship binaries that can be added to my path easily #7874

Closed
sethladd opened this issue Jan 12, 2013 · 20 comments
Assignees
Labels
P1 A high priority bug; for example, a single project is unusable or has many test failures type-enhancement A request for a change that isn't a bug
Milestone

Comments

@sethladd
Copy link
Contributor

When I install a gem from ruby today, and if it has binaries/scripts, they get added to my path. This is awesome. I'm sure I did a one-time setup but every other gem can supply util scripts that I can run easily.

I wanted to ship a simple util written in Dart, and it would be awesome if I could use pub to ship it. I would need pub packages to dump (or link or whatever) everything in bin/ to some well-known location that ends up on my PATH.

I searched and didn't find this issue, so here we are. Sorry if it's a dupe, I know we've talked about this before.

@munificent
Copy link
Member

This is definitely on our radar but thanks for making it official.


Set owner to @munificent.
Added this to the Later milestone.

@munificent
Copy link
Member

Issue #8281 has been merged into this issue.


cc @nex3.

@munificent
Copy link
Member

Issue #11785 has been merged into this issue.

@munificent
Copy link
Member

Issue #8815 has been merged into this issue.

@DartBot
Copy link

DartBot commented Nov 27, 2013

This comment was originally written by dangling.fe...@gmail.com


This example project demonstrates how this can be implemented in package manager.

http://pub.dartlang.org/packages/package_installer

The automatic execution of the process of installation of packages that required for their proper operation before they begin to be used. These processes of installation may include the following tasks: compilation of the binary files, installing the third party software and so on.

And this example demonstrates how to avoid requirements to include the binary files for all operating systems in package.

https://github.com/mezoni/demo_native_extension_usage

As the result of work of this example package that requirey compilation of binary files will be compiled and shared (it never be re-compiled when re-installing).

Bonus.
Not requires to writing batch files for each operating systems. The command batch files for compilation tasks can be replaced by single file written in Dart language (very convenient configuration).

http://pub.dartlang.org/packages/ccompilers

A set of classes for easy access to the system С/С++ compilers from the Dart language scripts.

This tools is a library written in Dart programming language. It is intended to compile C/C++ source files from Dart scripts.

@nex3
Copy link
Member

nex3 commented Jan 29, 2014

Issue #5473 has been merged into this issue.

@DartBot
Copy link

DartBot commented Mar 4, 2014

This comment was originally written by nerdre...@gmail.com


Is anyone working on this? Is there a plan for how this should be implemented? Is it on the roadmap?

@DartBot
Copy link

DartBot commented Mar 4, 2014

This comment was originally written by yulian...@yuvaks.com


This is one of the biggest features preventing me from writing half the things i want in Dart. I write them all in Node.js because node has simple package.json entries to create binaries from you main file, or provide paths to custom ones. or just wrappers scripts. The key is there is a global bin that lets you install it as an app.

This is a very important thing that many have asked for, and still being ignored. It can't be that hard to add.

@DartBot
Copy link

DartBot commented Mar 4, 2014

This comment was originally written by nerdr...@gmail.com


@seth: Funny enough, my name is also Andrew, and I thought your comment was
directed at me. I was thinking of working on a PR (or at least exploring
how difficult it is), but I wanted to make sure I wasn't wasting my time
before starting. Are there thoughts on how this should be implemented / the
pubspec.yaml changes?

Here's how npm does it: (quoting from
https://www.npmjs.org/doc/files/package.json.html)


To use this, supply a bin field in your package.json which is a map of
command name to local file name.
On install, npm will symlink that file into prefix/bin for global installs,
or ./node_modules/.bin/ for local installs.

For example, npm has this:

{ "bin" : { "npm" : "./cli.js" } }
So, when you install npm, it'll create a symlink from the cli.js script to
/usr/local/bin/npm.


And Rubygems: (quoting from
http://guides.rubygems.org/specification-reference/#executables)


Executables included in the gem.

For example, the rake gem has rake as an executable. You don’t specify the
full path (as in bin/rake);
all application-style files are expected to be found in bindir. These files
must be executable Ruby files.
Files that use bash or other interpreters will not work.

Usage:

spec.executables << 'rake'

@sethladd
Copy link
Contributor Author

sethladd commented Mar 4, 2014

Nerdrew, sorry for the confusion. I don't know if there's a design document for this feature yet. Sounds like you've just started one in your comment in #­14. :) Maybe start a Google Doc and share it with the community? We could ask the pub engineers to take a look at your doc to see if we're on the right track, as they probably have some context and thoughts.

@munificent
Copy link
Member

Are there thoughts on how this should be implemented / the
pubspec.yaml changes?

Yes, we've given it a bunch of thought but haven't had any time to work on it.

The pubspec part of it is actually easy: there's probably nothing to do there. Since we already have guidelines saying that command-line apps go in bin/, we can just assume all of the binaries there are the ones that should be made available.

The challenge is that there are a bunch of other issues to deal with. Say you have a package "foo" that has a single file bin/foo.dart that you want to make available. Here's a non-exhaustive list of the stuff that comes to mind.

* If you run a command from within a package that depends on "foo", it should use the version of "foo" that that package depends on.

  • It needs to handle that package depending on foo but not having foo in its lockfile.
  • It needs to handle the package depending on foo, having a lockfile, but not having installed foo already.
  • If you run the command outside of a package (or within a package that doesn't depend on foo), we need to determine which version of "foo" to use.
  • That probably means we need some idea of a "global" or "active" foo version, and we'll need a user experience to let the user configure that.
  • We need to make sure it works with things like path dependencies where "foo" may not be in the user's system cache.
  • Since "foo" just has a .dart file, we'd like to wrap that in something so that the user doesn't need to type ".dart". That means we need to locate the Dart executable. What's the best way to do that?
  • How do we handle "package:" imports in foo? We currently rely on symlinks for that, but we're trying to move away from that.
  • How do we ensure foo's transformers, and those of its dependencies get run when foo starts up?
  • If foo wants to load data files from within its own package, how does it locate them? We don't have anything like PATH or FILE in Dart.
  • We have to make sure it works in Linux, Mac, and Windows.
  • We have thorough integration tests for pub, so all of this will also need detailed tests.

I'm probably forgetting a bunch of stuff.

We'd love to have more external contributions to pub (and I can probably point you to a few bugs if you'd like), but there's a reason we haven't started working on this. It's quite complex, and it depends on a bunch of other stuff we are working on right now like transformers.

I can tell you what our rough plan is:

  1. We'll add a "pub run" command that invokes a named binary from a package. It can handle selecting the right version, running the transformers, etc. Similar to bundle exec.
  2. To let users just type command names without having to type "pub run" everytime, we can consider creating little shell wrappers that forward to that, similar to binstubs.

I don't mean to spook you here. I think it's awesome that you'd like to help out, and I can definitely help you get up to speed! But this is one of the most difficult corners of pub for someone (internal or external) to take on.

@nex3
Copy link
Member

nex3 commented Mar 4, 2014

Some additional thoughts:

That probably means we need some idea of a "global" or "active" foo version, and we'll need a user experience to let the user configure that.

I think this can come later; defaulting to the most recent version is probably enough for a first take on this.

Since "foo" just has a .dart file, we'd like to wrap that in something so that the user doesn't need to type ".dart". That means we need to locate the Dart executable. What's the best way to do that?

Pub's sdk.dart can tell us this.

We also don't want to require root privileges to install executables. This means we'll need to have some way of explaining to users on each platform how to add a directory within their .pub-cache to their PATH.

@DartBot
Copy link

DartBot commented Mar 4, 2014

This comment was originally written by yulian...@yuvaks.com


We also don't want to require root privileges to install executables. This means we'll need to have some way of explaining to users on each platform how to add a directory within their .pub-cache to their PATH.

Not necessarily. On OSX, /usr/local/bin seems to be open for writes without sudo. I know Homebrew uses it. Unless i missed a detail.

Also if I remember right, Linux has the same thing, since we use it here at work. I think its even the same location.

It is usually already part of the default path.

For Windows, setx does not need admin rights to set the path. so you can do "setx path "%path%:C:\Some\Path\To\Pub\bin"

@DartBot
Copy link

DartBot commented Mar 4, 2014

This comment was originally written by yulian...@yuvaks.com


With my last comment, I am trying to say it can be done by pub or assumed in default configurations.

@nex3
Copy link
Member

nex3 commented Mar 4, 2014

On my Linux box, /usr/local/bin is not user-writable. I don't think it's safe to assume otherwise, nor is it safe to overwrite other executables with conflicting names.

@DartBot
Copy link

DartBot commented Mar 4, 2014

This comment was originally written by yulian...@yuvaks.com


Good point. Maybe prompt the user to add a PATH variable to their .bashrc or .zshrc? or .profile? they are user controlled and will add the new path. It will function much like the Windows setx.

It can be like "We have detected that you do not have the Dart Bin path added to your PATH. Would you like us to add that for you?" and you do it only on the first ever pub run or something.

@DartBot
Copy link

DartBot commented Mar 30, 2014

This comment was originally written by martinchar...@gmail.com


I have created a design document for this feature. You can find it here

@munificent
Copy link
Member

@munificent
Copy link
Member

I'm retargeting this bug to be the umbrella bug for the more specific ones I'll be working on to support this.


Added Started label.
Marked this as being blocked by #18536, #18537, #18538, #18539.

@sethladd sethladd added Type-Enhancement P1 A high priority bug; for example, a single project is unusable or has many test failures labels Apr 30, 2014
@sethladd sethladd added this to the 1.7 milestone Apr 30, 2014
@DartBot
Copy link

DartBot commented Jun 5, 2015

This issue has been moved to dart-lang/pub#407.

@kevmoo kevmoo added type-enhancement A request for a change that isn't a bug and removed type-enhancement labels Mar 1, 2016
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 A high priority bug; for example, a single project is unusable or has many test failures type-enhancement A request for a change that isn't a bug
Projects
None yet
Development

No branches or pull requests

5 participants