Click here to monitor SSC

Michael Francis

Simple-Talk Exchange Editor

The Politics of Junk Filtering

Published Thursday, March 11, 2010 3:40 PM

If the national postal service, such as the Royal Mail in the UK, were to go through your letters and throw away all the stuff it considered to be junk instead of delivering it to you, you might be rather pleased until you discovered that it took a too liberal decision about what was junk.

Catalogs you'd asked for? Junk! Requests from charities? Who needs them! Parcels from competing carriers? Toss them away! The possibility for abuse for an agency that was in a monopolistic position is just too scary to tolerate. After all, the postal service could charge 'consultancy fees' to any sender who wanted to guarantee that his stuff got delivered, or they could even farm this out to other companies.

Because Microsoft Outlook is just about the only email client used by the international business community in the west, its' SPAM filter is the final arbiter as to what gets read. My Outlook 2007, set to the default settings, junks all the perfectly innocent email newsletters that I subscribe to.

Whereas Google Mail, Yahoo, and LIVE are all pretty accurate in detecting spam, Outlook makes all sorts of silly mistakes. The documentation speaks techno-babble about 'advanced heuristics', but the result boils down to an inaccurate mess. The more that Microsoft fiddles with it, the stickier the mess. To make matters worse, it still lets through obvious spam. The filter is occasionally updated along with other automatic 'security' updates you opt for automatic updates.

As an editor for a popular online publication that provides a newsletter service, this is an obvious source of frustration. We follow all the best-practices we know about. We ensure that it is a trivial task to opt out of receiving it. We format the newsletter to the requirements of the Service Providers. We follow up, and resolve, every complaint. As a result, it gets delivered. It is galling to discover that, after all that effort, Outlook then often judges the contents to be junk on a whim, so you don't get to see it.

A few days ago, Microsoft published the PST file format specification, under pressure from a European Union interoperability investigation by ECIS (the European Committee for Interoperable Systems). The objective was that other applications could then access existing PST files so as to migrate from existing Outlook installations to other solutions. Joaquín Almunia, the current competition commissioner, should now turn his attention to the more subtle problems of Microsoft Outlook. The Junk problem seems to have come from clumsy implementation of client-side spam filtering rather than from deliberate exploitation of a monopoly on the desktop email client for businesses, but it is a growing problem nonetheless.

Cheers,

Michael Francis

by mikef

Comments

 

web said:

Just what we need, the EU running American corporations.
March 13, 2010 7:42 PM
 

randyvol said:

Michael -

I'm a long-suffering Outlook user; having had it inflicted on me from the very top of the organizational tree.

I'd like to summarize some hard learned lessons in a response to your spot-on blog post which I will call "The Market-ics of Microsoft products";

Hard-learned lesson #1; End-users are busy with their 'real jobs' and won't be bothered learning an application unless forced to from the top-down.  Since they use Office and outlook at home (even if it is express), they will not learn another product for work unless it is a condition of employment.
Hard-learned lesson #1b; Executives are busy running businesses and won't be bothered learning an application - period  They use Office and Outlook at home; ergo it is what will be used at work.  Rule #1 what I believe Microsoft has abstracted from hard-learned lessons # 1 and 1b - Just market your application as 'intuitive'; respond to any security concerns with 'marketecture' claims such as 'comprehensive', 'bullet-proof';   Marketing Tactic #1, abstracted from Lessons 1 and 1b, and Rule #1 - your target market is NOT IT.  IT is the enemy.  Your target market is: Executives and Management; you messages are 'easy to use!',  'meets your needs' and 'is secure'
Hard-learned lesson #2; Having been circumvented in the decision-making process by Marketing Tactic #1; IT is left to account for its own inability to manage the most excellent solution that was purchased on the decision of Executives and/or Departmental Managers.  (ie. why is it that you have configured Outlook in such a way that it is blocking and junking email I want to receive?) Hard-learned lesson #2b; If 'everyone' is using it, then it must be the best thing for us as well.  Apparently, unlike me, Executives and Departmental Managers never had a parent that asked them 'If everyone were jumping off the Empire State building, would you do that as well?'.  So, the logical conclusion is that if everyone buys it, it is the best, therefore any problems must be because IT is not up to the task of managing 'the best'.

Summary of hard learned lessons - the market works very much like a democracy, and as we like to say in the States, 'the terrible thing about a democracy is you get what you deserve'.  The terrible thing about Outlook is that business is getting the flawed email system it deserves.

I can never understand why companies will invest months and months and months examining competitive offerings for say, an ERP system, but will just make a knee-jerk 'buy Microsoft' decision on other things like Office productivity and eMail.

I once was given the marching order to find an email system that 'does everything Outlook does, but doesn't cost as much'.  Well, those products are out there.  The one I found did everything Outlook did at the time, plus much more and was less expensive to boot.  It provided deep and manageable heuristics for junk mail as well as BOTH black and white lists that could be managed at a global level and expanded/modified at individual mailbox level; it had calendaring and tasks, etc.  In short it worked perfectly and met the requirements given IT by management.  In less than a month, it was rejected by the President, because... wait for it -

It is not Outlook.

Which leads me to Hard-learned Lesson #3; As a defense tactic, always document the requirements for any IT solution.  Get the top of the organizational tree to sign off on it as the 'contract' between IT and the company.  When the requirement consists of 'we want Outlook' then the response can be - you wanted Outlook, you have Outlook, and Outlook's deficient junk email handling is defacto a stated requirement in the contract.

It is not the most ideal response; but in a world where some acquisition decisions are really not based on the best of breed solution to a set of requirements, rather they are an acquisiton based on brand, then the answer must perforce always be 'You have exactly what you asked to have; warts and all.  I cannot expend my few and precious budget dollars trying to fix Microsoft's deficiencies.'  I always try to end this type of response with a polite 'Is there anything else I can do for you?'
March 15, 2010 11:36 AM
You need to sign in to comment on this blog

About mikef

Michael Francis is the Simple Talk Exchange Editor. Michael has 15 years’ experience in editing, writing, and marketing covering a broad spectrum of topics and publications. He has edited technical patents and chemistry journals, written for publications ranging from New Scientist to Pest Control News, and marketed scientific modeling software, machine-to-machine connectivity, and SQL Server and Exchange Server tools. In his spare time Michael enjoys cricket, natural history, camping, and getting beaten at football by his children.
<March 2010>
SuMoTuWeThFrSa
28123456
78910111213
14151617181920
21222324252627
28293031123
45678910
How to Kill a Company in One Step or Save it in Three
 The majority of companies that suffer a major data loss subsequently go out of business. David Wesley... Read more...

Migrating from OCS 2007 R2 to Lync: Part 4
 Having migrated the rest of our users and legacy resources across, and start getting ready to... Read more...

Automated Script-generation with Powershell and SMO
 In the first of a series of articles on automating the process of building, modifying and copying SQL... Read more...

Seth Godin: Big in the IT Business
 Seth Godin has transformed our understanding of marketing in IT. He invented the concept of 'permission... Read more...

Using SQL Test Database Unit Testing with TeamCity Continuous Integration
 With database applications, the process of test and integration can be frustratingly slow because so... Read more...