Tony Davis

Simple-Talk Editor
News, views and good brews

LINQ and vNext

Published Thursday, November 15, 2007 11:13 AM

The history of IT is littered with the corpses of software companies that assumed that a solution that worked well in particular circumstances would necessarily become a generic panacea. Who, for example, now remembers the past giants of the Workflow industry?

The Orcas chrysalis is now hatching into the Visual Studio 2008 butterfly. Well, most of it. Visual Studio 2008 and .NET FX 3.5 will ship before the end of November. The Entity Framework, vNext, isn't ready yet. There is a lot in Visual Studio 2008, and it will take time to assimilate. Language Integrated Query (LINQ) has had a lot of attention. It is a neat solution, but not to every problem. The concept is difficult to argue with: we have just one querying language for every source of data. Data sources can export their metadata to the application; we then create logical, object-oriented, data sources by mapping entity-relational models to the object model (which is where vNext comes in). The problems of having to learn a different query language for every data repository, XML, LDAP, array, enumerable class, dataset, in-memory object, relational database, document, ISAM (…and so on) suddenly vanishes.

However, what works well with an XML file, a dataset, or a low-volume data-store could, in fact, be a disaster for a commercial, high-volume online transactional processing system. The problem is that the cost of a harmonised and logical application interface is likely to be a cacophony of 'dynamic SQL' at the database.

It is true that LINQ and vNext Entity Framework can work with stored procedures, but recent Microsoft presentations have made little of this. In fact, the difference in tone at PASS and TechEd was striking, with the emphasis to the developers being much more based on making direct calls into the database at table level. If this comes about, then there will be interesting times ahead for the DBAs. The problems of tracking and correcting locking/blocking problems, rogue queries, performance problems due to poorly constructed queries or the flushing of the procedure cache, not to mention security issues, will keep them busy; so busy, they won't have the time to worry about the fact that they cannot change the entity/relationship model without breaking LINQ code.

vNext is probably going to be the vital technology for the data architect because it provides the means to present one rational Entity Data Model for the application, rather than maintaining separate EDMs for persistence, reporting (e.g. SMDL), analysis (e.g. UDM), workflow, or data management (e.g. SMO).

It seems likely (though at this stage, not certain) that one will be able to provide a logical interface between the EDM and SQL Server that is based on stored procedures, views and functions. The application developers will be able to LINQ into the data model, thereby avoiding what they refer to as 'impedence mismatch'. This looks a lot more like a solution that will please everybody.

So what are your thoughts on this contentious LINQ issue? Just add them as a comment below (you'll have to be singed in). Again, all contributions will go into the draw for an iPod Shuffle!

Cheers,

Tony.

 

 

Comments

 

ASdanz said:

Delivering LINQ without vNext is just throwing fuel on the Developer vs. DBA fire that is smoldering almost everywhere. . There are enough developers in the world that think: "DBA's? We don't need no stinkin' DBA's!"
November 15, 2007 8:04 AM
 

glenfitch said:

I'd certainly feel a lot happier if we were getting warm reassurance that one could use stored procedures with the Entity Framework, vNext. Pablo writes....

'The way we're thinking of store-procs support for fetching data is more along the lines of defining well-known, fixed entry points that map to stored-procedures in the store. You wouldn't be able to formulate queries over them, you would only be able to invoke them using some API or syntax construct.'

I'm still pondering that one.
November 15, 2007 8:42 AM
 

EntArch said:

The issue with Microsoft with it's recent tools, frameworks, etc is that it chooses to highlight the "cool" factor without providing any guidance or best practices. And developers, being developers, want to adopt these features without understanding them since they are REA (Resume Enhancing Acronyms).

I lay the blame entirely at the Microsoft Developer Relations (DRG or whatever it's called now) for:

1. not being ready with guidance before the tool is released
2. being selective in what is being touted.

My $0.02 ... probably less due to (true) inflation.
November 15, 2007 10:32 AM
 

david.buckingham said:

Having worked on two systems in the past that were almost exclusively run-time generated dynamic SQL, I fear any application based on LINQ that is destined for many simultaneous users --- if it is direct-to-table queries vs using stored procedures.

Security, locking/blocking, performance tuning, etc. will be a nightmare for the unfortunate DBA.
November 15, 2007 1:35 PM
 

normancmore said:

By default, vb.net 2003 used stored procedures when you 'configured' a dataset. In .net 2005 it defaults to using direct sql statements.

As the above statement indicates, the fight is being won by the side that promotes writing quick things that work short term. Theory and best practices are undermined by the very companies (i.e. Microsoft) that tout them. Is this right? Considering the unpopular fact that most applications will NOT have a high transactional volume since most business endeavours fail, then yes it is right.

Unfortunately, being right or wrong isn't the issue. If you say something ugly, it will impact your job negatively. Nobody paying you to implement their latest bright idea is going to appreciate where they fit in the big picture. You still have to sound like you are aware of all the latest breaking news, even if it doesn't make any sense. You have to sound like their latest idea will be the next successful venture. In other words, you have to contradict yourself just like Microsoft does.
November 15, 2007 2:01 PM
 

eliteab » Blog Archive » LINQ and vNext said:

November 15, 2007 3:08 PM
 

Mike Gale said:

I really can't see Dynamic SQL being a solid engineering solution.

We knew about the dangers of this stuff years ago.
November 15, 2007 10:28 PM
 

Raju said:

Software is one industry where every release is expected to solve all problems, but every release adds new problems.
November 16, 2007 12:48 AM
 

Sean Kearon said:

I'm not sure that dynamic SQL generation always needs to be a problem.  We do that here, caching the statements and using paramaterised SQL - it's very smooth and we don't have to write data access layers in our apps.  Then also cache the business objects in a cache layer to save getting the item from the DB each time.  

If I had time to rewrite, I'd use either LINQ or NHibernate for the data access layer.  Speaking of NHibernate....can we really say that an ORM layer is not performant and scalable???
November 16, 2007 1:58 AM
 

Steve S said:

It's very easy to criticise anything you don't understand, and of course there are many sides to every argument.

LINQ is one of the few genuine innovations coming out of Redmond at the moment and for many will result in writing less code, better code, and sometimes, even more efficient database queries. There will still be a place for fine hand-tuning in special cases, and the API supports that through stored procs cleanly.

80% of the data access code written by both developers and DBAs is pointless repetition that can be eliminated, so we can focus on the critical 20%. Hopefully, LINQ is going to help us out here.
November 16, 2007 3:27 AM
 

Steve S said:

Also, as Tony pointed out, the Entity Framework could end up giving more flexibility and control to DBAs as it separates the physical model as seen by the database administrator from the logical model that developers are working with. This may mean conflicts of interest can be resolved via changing the mappings, not the database or the apps.
November 16, 2007 4:20 AM
 

paschott said:

I agree with a lot of the above comments.  LINQ will be cool and the idea of reducing some of the horrible Dynamic Generated SQL I've seen.  However, we'll get away from using stored procedures appropriately and we'll definitely have more issues trying to fix SQL code or make it run better.  I've been fighting with the dynamic SQL problems for a while now and every time it's something buried in our C# code, I have to push it over to a developer to eventually look at it and find out what's actually being done, then wait for them to update the code, then recompile, test, push through the environments, etc.  Tweaking a stored proc?  Make change, test, push through environments quickly without needing to recompile and worry needlessly about branches.

The good thing is that I'm able to develop the more SQL-intense queries inside of stored procs or explain why something might be better off in code or in SQL.  Some items don't matter too much, but returning a large set of data and operating on that set is almost always better through a stored proc call rather than trying to work on a set row-by-row.

I will have to read up on LINQ a little more.  If changing the underlying structure will break LINQ, that could be an issue.  I remember tightly-bound tables with FoxPro where adding a column broke the entire app.  I really hope LINQ doesn't do that - I can't imagine that it would, but that possibility scares me.  We also need to figure out how to set the TRANSACTION ISOLATION levels - we use a lot of READ UNCOMMITTED calls because we don't care too much if the data is dirty in most cases for read and it really helps reduce locking contention.  I realize that option isn't right for everyone, but it helps us.
November 16, 2007 9:11 AM
 

Daniel Penrod said:

And to add to the mix MS is developing the "Ruby on Rails" methodology with their MVC framework.  

Scott G writes:
"The ASP.NET MVC Framework enables you to use any data access pattern or framework you want in order to retrieve and manage your models.  If you want to use ADO.NET DataSets/DataReaders (or abstractions built on top of them) you can.  If you prefer to use an object relational mapper (ORM) like NHibernate, LLBLGen, WilsonORMapper, LINQ to SQL/LINQ to Entities you can absolutely use those as well."

The good thing is we aren't forced to use LINQ if we don't want to.  The DBAs can still be happy if we (as developers) are willing to compromise and do it another way.  There is more ways than one to skin a cat!
November 16, 2007 9:24 AM
 

marmot4 said:

I think the whole EDM thing is very intriguing.  By abstracting out the details of the database schema you could save your application developers some time.  I'm interested to learn what performance concerns such an abstraction layer might introduce though.  I'm also far from sold that abstraction for the sake of abstraction is a good thing.  If it saves me time when dealing with complicated schemas, I'm on board.
November 16, 2007 10:53 AM
 

Dave S said:

LINQ is indeed a great technology for developers that has a very practical purpose like Steve S said, resulting in writing less code.  And like Daniel Penrod said, we're not forced to use LINQ, but we do want to take advantage of the benefits LINQ offers.

However, some of the issues mentioned here have me concerned.  If there is more emphasis on creating direct calls to database tables and using dynamic SQL statements, us DBA's are absolutely going to have our hands full.  It seems that somebody is already are or will eventually become aware of the issues mentioned here and come up with the solution that works with stored procedures, views, and functions.  We DBAs shouldn't allow any more ad-hoc queries directly accessing our tables than necessary, mainly because we're already too busy dealing with the security and performance issues from the older applications that already do that.

I liked this thread, it really got me thinking.
November 16, 2007 10:58 AM
 

Tony Davis said:

All,

Many thanks for all the interesting and thoughtful contributions to this thread. I have just performed the draw and the winner of the iPod Shuffle is...

paschott

Best Regards,

Tony.
November 30, 2007 11:14 AM
 

http://www.simple-talk.com/community/blogs/tony_davis/archive/2007/11/15/39701.aspx said:

April 5, 2008 8:54 AM
You need to sign in to comment on this blog

















<November 2007>
SuMoTuWeThFrSa
28293031123
45678910
11121314151617
18192021222324
2526272829301
2345678
Investigating SQL Server 2008 Wait Events with XEVENTS
 Some reasons for the slow-running of database applications aren't obvious. Occasionally, even the... Read more...

Controlling Email Messages using Exchange's Transport Rules
 Some tasks that should have been easy in previous versions of Exchange just weren't. Now, with... Read more...

Software Tool design: The Three Rs
 To understand the full extent of the requirements of your users when you are redesigning a software... Read more...

JSON and other data serialization languages
 The easiest way to speed up an Ajax application is to take out the 'X' and use JSON rather than XML. Of... Read more...

Embedding Help so it will be used
 It is not good enough to make assumptions about the way that users go about getting help when they use... Read more...