New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
performance of addAll when mixing in ListMixin #12652
Comments
Not clear to me whether this has been resolved on the list or not. Added Area-Library, Triaged labels. |
We have found a solution, but it might be interesting for the VM devs to investigate if there are any low hanging fruits for the 50x factor. The mixin implementation just sets the length to length+1 and then assigns into the new slot. My expectation would be that this is slower than the native addAll but not 50x. cc @fsc8000. |
cc @sgmitrovic. |
Removed Priority-Unassigned label. |
VMs growable array length setter does not grow efficiently: In contrast, 'add' method: |
The performance issue would be fixed if we change ListMixin.addAll to something like: We have decided that VM library's length setter should not grow the list aggressively as we would loose the ability to control the memory growth. cc @floitschG. |
We generally prefer not to run through the iterator twice. We could call iterable.toList() and then use the list instead, but that would cost in memory. |
I.e., this is another case where we could do better if we knew that .length was efficient, but we don't, so we can't. Testing for List is a hack which misses a lot of other good length-known collections. |
To clarify: .length is quite efficient. It is the growing and the included copying of the data that is expensive. In the suggested solution you would still be much faster with this[this.length++] if you primed the capacity of 'this' by setting it to the maxed size once: void addAll(Iterable<E> iterable) { I think this is an example where sharing code is bad as it includes significant performance penalty: some iterables have expensive length increase past the capacity, some iterables have expensive length computation. That is why the user must override certain ListMixin methods (as VM-s library do) in order to get acceptable performance. Maybe this bug should be marked as 'Works as designed'? |
I was referring to the .length of the Iterable argument to addAll. When I talk about an "efficient length", we have considered (but never really accepted) adding a "fastLength" getter to Iterable that will always return in constant time, but may return null if it can't find a result that fast. That way we would be able to know in advance how many elements the iterable has, if the iterable knows it itself, and still leave room for lazy iterables that require a full traversal to know the length. It's not pretty, though. One solution for the current problem could be to increment length by more than one (e.g., double it before starting, and then setting it back to the final count when we are done). |
Let's document this case in the dartdocs of the mixin and close this bug. Added AsDesigned label. |
Created issue #12840 to track the documentation update process. |
This issue was originally filed by @Bluenuance
creating ones own list which simply changes a bool flag when something changed in the list using the ListMixin like this:
class MyList<E extends num> extends Object with ListMixin<E> {
List<E> _list = new List<E>();
bool mybool;
E operator [](int index) => _list[index];
void changed() {
//do something extremely cheap as
mybool = true;
}
void operator []=(int index, E value) {
_list[index] = value;
changed();
}
int get length => _list.length;
void set length(int newLength) {
_list.length = newLength;
changed();
}
}
now perform an addAll method with quite some elements using something like this:
toBeAdded = new Iterable.generate(10000, (i) => rnd.nextDouble()).toList(growable: false);
it takes (not in checked mode) about 130ms in comparison to a "normal" list which takes around 2ms! (cannot be the changed() method, as when removed , nothing 'changes' performance wise)
now when I overwrite addAll with:
void addAll(Iterable<E> iterable) {
_list.addAll(iterable);
change();
}
I'm at about 1 to 3ms
this was (nearly) pasted from the forum:
https://groups.google.com/a/dartlang.org/forum/?fromgroups#!topic/eng/1YZTtIS9kSk
The text was updated successfully, but these errors were encountered: