I guess if it were a really big egg, you could dig a little ditch going from the hen-house to your kitchen, break the egg, and stream it into the kitchen. Might need to make sure you could cook it fast enough, though - a buffer overflow could be messy!

---

Robert Chipperfield
Developer
Red Gate Software Ltd

As many have pointed out, the answer should be "it depends".

While others have posted about some detailed technical issues that will influence performance, for most cases, the overall impact on system application performance may not even be measurable, let alone significant enough to impact performance.

What we use to determine which construct to use is often the skill level of our client companies development staff who will be living with the code for years.

If you are given a choice between the following two scenarios:

Scenario A:

* Code is implemented, and bug free in 1 hour by a junior developer
* Code is easily understood by all of the maintenance team.
* Code requires 50mS to execute in response to a button click.

Scenario B:

* Code is implemented, and bug free in 2 hours but required a senior developer
* Maintenance Team des not fully appreciate the implementation, and introduces a bug that causes an update to miss it's ship date..
* Code requires 10mS to execute in response to a button click.

If "Time is Money"....which is the faster code????

Our policy is to have a performance SPECIFICATION and performance testing from the very early phases. Provided the implementation meets (with a reasonable saftey margin) the specification, then the prime directive shall be the maximizing of Reliability, Maintainability by the broadest base of the avaiable resources. Sacrificing either of these for (unnecessary) performance gains is a losing proposition.

 

---

TheCPUWizard is a trademark of Dynamic Concepts Development Corp. Please visit us at www.dynconcepts.com

Yield every time.

Locality would be the main benefit of pre-computing the list (setting aside concerns on size outstripping available memory) as both generator and consumer code will be able to benefit from better cache-hit rates.  Lazily evaluated collections may not get the same benefits as both the generator and consumer code get interleaved and so will probably end up accessing main memory more often - individually small costs, but they add up.

Pre-computing would probably also be better from an object-lifetime perspective - if either the generator or consumer has some expensive objects, they will be open and using resources for longer...


Saying that, I would write the generator function using yield everytime - the code is easier to write, comes out easier to read and debug, and will run just as quick as pre-computed collections in most "real-world" situations.  If performance analysis shows there is a case where pre-computation would be best, Linq provides such methods as ToArray() and ToList(), which can get you a precomputed collection with no more effort on your part (plus, you get benefit of deciding which approach is best for a given situation)

Thought I'd add a couple of extra thoughts which aren't directly perf-related...

Reuse: If the generated collection will be static and can be safely reused by other consumers, precomputation and caching might make more sense.

Mutation: If you just populate a List<T> and return an IEnumerable<T>, unless you are also wrapping it in a ReadOnlyCollection or similar, a consumer could still cast it to List<T> and modify the values.  This is made worse if combined with the Reuse point above.

Concurrency: The amount of effort required to write efficient (faster than just using ToArray) hand-coded pre-computed-enumerator code will probably be greater than that required to convert it to multi-threaded - especially so if will be using the upcoming parallel extensions framework.

There are also benefits to lazy evaluation (which have been mentioned earlier) in situations where only a partial recordset is required (e.g. only the first 100 records) and where the [possibly expensive] resources required by the generator function are only instantiated upon first access.

V good points Lewis. Setting aside the disturbing thought of a hen that lays eggs bigger than me, I guess I'd just have to take a list the eggs home. I'd set up some off-line processing to cook them in situ, and order a managable morsals from the safety of my kitchen. To keep it secret I could always have them scrambled first.

I got the news letter today and come here immediately. Many experts have talked, but I'd like to throw in my opinion.As far as I can understand, the question is about processing some data into a list, I think it must be better to build up collection and return because yield will turn to multi-threading invocation (Active Object Pattern, if I'm not wrong?), which will cause overhead of extra code execution.
So, I vote for build up a collection.

Maybe I'm over-simplifying, but isn't the code that uses yield only going to be faster if the caller of the code that yields does not loop over the whole list? In the end, isn't the caller of the yield code going to enumerate over each item in the source list anyway? Then the only cost savings is that you're not calling List.Add. Maybe I'm making too many assumptions about the caller, but it sounds like I'm in the "it depends" camp.

"It depends" is a good position.

If you are concerned with memory footprint, but can afford time cost, go with yield.

If you are making explicit usages of either a ringbuffer setup or deferred execution, go with yield.

If you are reusing data, preprocessing, casting lists, bulk processing etc, go with returning collections.

 

If it makes sense within the object itself that the objects returned will appear in sequence and in realtime, I think I'd consider going with yield anytime. However, the point about readability is important.

And then ANTS Profiler can teach me otherwise :)