Inside the tent....

DevOps: Nostrums or Knowledge?

Andrew Clarke — Tue, 16 Aug 2011 00:25:00 GMT

There are good reasons for the management of the release of applications. Businesses see it as a safety-net to ensure the success of software deployment. This is a process that requires a different mind-set and set of disciplines to development, and is best handled by small specialist teams that are responsible for getting software delivered to its users in an enterprise. It is meticulous work, because users, and the businesses that employ them, judge software primarily by its resilience: They care a great deal if a release goes wrong and errors get into production systems.

Whereas the development cycle has speeded up greatly in the past decade, under the weight of new development techniques, and pressures from the business for new functionality, the same is not so true of release. Developers are often shocked and puzzled by the innate conservatism of the release Management process, but the application delivery process has to be meticulous enough to prevent errors getting into production systems. It has to deal with a wider range of platforms, including the Cloud, virtual servers, and mobile devices, and a more complex configuration.

This has led to a problem that has afflicted IT departments that have adopted the ideal of the rapid development cycle and Continuous Integration. How does one release applications to the users more quickly in line with the increased speed of development? Whereas one might think that it would require the same effort to manage a small number of changes in a large number of releases as the other way around; it doesn’t look that way to those of us who are tasked with release. A release is a release, no matter how numerous or complex the changes.

DevOps has never been presented as a nostrum. It doesn’t involve group rituals such as scrums and ‘post-its’ on the wall. (Rapid Development used to involve ‘shirtsleeves’, large felt-tips and A2 sheets fastened on the wall) It is more about Dev and Ops working in collaboration instead of seeing the opposite camp as being adversaries, defining the most effective workflows, and getting the processes reviewed and refined. All this means a much greater coordination.

At this point, automation of at least part of the release process becomes possible. There are prime candidates for automation, such as ‘hot-fixes’, data-center deployments, and configuration management. However, without the multi-department collaboration that is essential for the rapid delivery of applications, automation can have the danger of merely allowing mistakes to be made faster. The Automation process has to be scripted, maintained and controlled by the Ops and QA staff themselves, rather than being created only by developers. Whilst it is an essential component in the devOps initiative, it is secondary in importance to the cultural and organizational changes that often have to take place before continuous deployment or continuous delivery can become a reality.

The Cookie Crumbles

Andrew Clarke — Mon, 04 Jul 2011 21:34:00 GMT

Cookies were never intended to invade your privacy. Transient session cookies were invented out of necessity, by Lou Montulli at Netscape in June 1994, purely to make the use of a shopping cart possible on the stateless web. Permanent, or tracking, cookies soon followed, in order to identify users between sessions and so save users the tedium of having to identify themselves for every session. Such cookies should be innocuous because they can only be read by the site contained in the cookie. Nevertheless they were, even then, viewed with suspicion, as a security risk, since if the cookie can be somehow copied it can used to impersonate the user.

The biggest problem with persistent cookies, however, is that they can contain 'third-party' domains, rather than just the domain of the site that writes it. A 'third-party' domain is easily written to from any site that chooses to do so, allowing an unscrupulous marketing agency to collect information about the browsing and buying habits of internet shoppers, across all the sites where it has its advertisements or web bugs placed, and so target advertising to the individual. Although the US government has strict rules against the use of persistent cookies in this manner, the same isn't true of commercial sites, which have resisted voluntary regulation of the use of third-party cookies.

Due to widespread concerns about this invasion of privacy, the "European Commission Privacy and Electronic Communications Directive" was issued, and has to be implemented by every member state by May 25th. (So far, only the northern European countries have complied). It changes the requirement that the user has a right to refuse to store cookies, either third-party or not, on a local machine, to an obligation on the part of the website to give explicit "informed consent" on all cookies being used, even the session cookies.

This is a botched reaction. The legislators seem to have confused simple cookies and third-party cookies. The Directive allows cookies only for activities that are 'strictly necessary' for the operation of a website and its delivery of those services that a user has explicitly requested. This is open to varying interpretations, and the UK government has concluded that 'we consider that it will, for instance, cover (allow) the use of cookies that underpin the use of shopping baskets on websites.' However, the law was passed together with the ambiguous wording.

The IT industry foolishly concluded that, if the user's browser settings allowed third-party cookies, then they have given consent to them. Not so, said the EU, and since then the IT industry has been negotiating with the EU and member states on best practices that would be considered compliant. The UK came up with a 'best practice' consisting of "…an easily recognisable internet icon, a privacy policy notice, a single consumer control page, with a self-regulatory compliance and enforcement mechanism…", via which a consumer could access details about each specific internet advert, the advertiser, the server, and so on, and refuse the cookie, if desired.

This is all very well-meaning, but also very silly. Cookies are necessary, and very few users will tolerate having to click to opt-in on every site they visit. It's also unlikely that users will check the details of every advert before opting to allow cookies for a site, and the site containing the advert could be completely unaware of the data being collected. Therefore, it will hardly prevent an unscrupulous marketing organisation from harvesting the users' internet activities.

As if to prove that the 'e-privacy' directive is fatuous and unworkable, tracked traffic to the website of the Information Commissioner's Office (ICO) fell by 90% when it recently adopted measures to gain cookie consent. A freedom of information (FOI) request by Vicky Brock, a Web Analyst forced them to release the information.

This is a graph that is likely to strike fear into any well-meaning site that tries to comply with this regulation.

Surely, a much more sensible solution is this: Browsers shouldn't allow third-party cookies by default, as they serve no honourable purpose; though, for some reason, Microsoft's Hotmail, MSN, and Windows Live Mail webmail require them!. Currently, users have to explicitly opt out, by turning off third-party cookies. This simply needs to change;

Instead, all browser publishers seem hell-bent on making it more difficult to opt out of third-party cookies; it took me ten minutes to discover how to do it in Firefox. It would seem that the power of the advertising and marketing interests on the Internet are too powerful to ignore, and instead we are likely to see, on every site, tedious opt-in forms and ‘easily recognisable internet icons, privacy policy notices, consumer control pages, and other mandatory gubbins. How much more sensible would have been a voluntary code of practice.

Cloud Cuckoo Land

Andrew Clarke — Tue, 10 May 2011 00:37:00 GMT

Does it strike you odd that, despite the outage of Amazon Elastic Compute Cloud (EC2) and Elastic Block Storage on April 21 2011, many of the customers who were affected are still so keen on the service?

It puzzled me until I considered the huge diversity of applications out there. If these customers had been trading eCommerce sites, 2-4 days of downtime would have been a severe blow. For a provider of financial services, even a day’s loss of services is unthinkable. The problems of the Millennium Exchange system on the London Stock exchange in February, for example, lasted only four hours, but caused a 30% drop in FTSE 100 turnover.

For other types of application, however, this isn’t such a problem, and herein lies the source of a major sticking point in the on-going debate about the usefulness of the ‘cloud’: we all extend our limited experience of commercial IT and generalize it beyond reason. We think that what was good for the projects we were directly involved with would be good generally, but it’s simply not true.

As the IT industry diversifies from its roots in commerce, taking on social interaction, gaming, gambling, armaments, manufacturing, espionage, publishing, politics, and so on, it becomes ever more fragmented.

A generation of developers, the ‘Dotcom-boom-babies’, has been able to work in IT with far less experience of the constituency where current dogmas and ‘best-practices’ first evolved. What is appropriate for developing applications that are critical to the health of international corporations, under the yoke of international legislation for handling and storing financial and personal data, would be curiously arcane for a ‘fluffy’, rapidly-evolving networking site. An architectural framework that suits a site like this is worlds away from the resilience required for a system that supports a major company;

Even within the corporates, there are huge differences that will determine IT architectural choices. The culture of IT in manufacturing is quite different from that in banking or the way that government IT is run. A manager of a major medical application once remarked to me that whereas a company might lose millions in revenue if one of my applications suffered more than a few minutes of downtime, he was kept concentrated on the fact that people would die if one of his applications failed.

The marketing men may bombard us with generalizations about visualization, the cloud, distributed architectures, agile development techniques, as if they were back to selling soap powder, but in such a diverse industry, one man’s meat is another man’s poison.

Virtual Irony for Oracle

Andrew Clarke — Tue, 12 Apr 2011 00:47:00 GMT

Where’s Oracle VM 3, and why should you be waiting for it? Here lies a puzzling story.

On May 13, 2009, in the course of developing Oracle VM 3, Oracle acquired Virtual Iron Software Inc and with it, their rather good VM product Virtual Iron. In June 30th 2009, scarcely a month later, they had ditched the product, saying “Oracle will suspend development of existing Virtual Iron products and will suspend delivery of orders to new customers”. It was a move that puzzled everyone at the time. Virtual Iron had been a US start-up with a Xen/Java server virtualization product based, like Oracle VM, on the open source Xen hypervisor. The bulk of Virtual Iron’s 2,000 customers were small- and medium-sized companies who were using it as a cheaper and simpler alternative to VMware. The product was, evidently, well-received by its customers who then, it seems, shrugged and moved to VMWare. Oracle seemed to have acquired Virtual Iron merely to bolster their Oracle VM team and to enhance the rather lacklustre Oracle VM 2.2 with Virtual Iron technologies. These technologies that attracted Oracle were Capacity and power management, an Open scriptable API, and a scalable, modular management framework.

We all thought that the effect of this cull of a promising company would be to accelerate the development of Oracle VM 3, predicted originally, and unofficially, for release in 2009 by Wim Coekaerts, vice president of Linux and virtualization engineering.

Sadly, it didn’t really work out that way. 2009 Oracle OpenWorld came and went, as did 2010 Oracle OpenWorld: No Oracle VM 3. Finally, It went into Beta in December 2010. And in February was reported by Herbert van den Bergh to be in the final production phase. At the time of writing, the best guess for its belated appearance is 2011 Oracle OpenWorld.

So why do we need Oracle VM 3.0? Whereas Oracle VM 3.0 could, once it arrives, persuade Red Hat users to move to Oracle for their Linux, virtualization and application server support, it is less likely to induce VMware/Oracle customers to migrate their Oracle workloads to Oracle VM 3.0. The aging Oracle VM 2.2 didn’t compare well with competing products regarding load-balancing; it has no snapshots, suffers tricky backups and cloning, and doesn’t have memory consolidation, so Version 3 would need to leapfrog the current industry standards. Even though we are assured by insiders that Oracle VM 3 will be the best VM product to host Oracle Database, there is a another problem for anyone with Microsoft Windows servers. Neither Exchange 2007 nor Windows 2008 are currently supported to run on Oracle VM. It seems that Oracle has no VM product that is certified to run Microsoft software at present. If you are aiming to have one VM solution for all your servers, including Oracle Database, you’re currently out of luck if you prefer it to start with the letter O.

The Hot-Add Memory Hogs

Andrew Clarke — Tue, 15 Mar 2011 01:42:00 GMT

One of the more difficult tasks, when virtualizing a server, is to determine the amount of memory that Hypervisor should assign to the virtual machine. This requires accurate monitoring and, because of the consequences of setting the value too low, there is a great temptation to err on the side of over-provisioning. This results in fewer guest VMs and, in fact, with more accurate memory provisioning, many virtual environments could support 30% more VMs.

In order to achieve a better consolidation (aka VM density) ratio, Windows Server 2008 R2 SP1 has introduced what Microsoft calls ‘Dynamic Memory’. This means that the start-up RAM VM memory assigned to guest virtual machines can be allowed to vary according to demand, changing dynamically while the VM is running, based on the workload of applications running inside. If demand outstrips supply, then memory can be rationed according to the ‘memory weight’ assigned to the guest VM. By this mechanism, memory becomes a shared resource that can be reallocated automatically as demand patterns vary. Unlike VMWare’s Memory Overcommit technology, the sum of all the memory allocations to each virtual machine will not exceed the total memory of the host computer.

This is fine for applications that are self-regulating in their demands for memory, releasing memory back into the 'pool' when not under peak load. Other applications however, such as SQL Server Standard and Enterprise, are by nature, memory hogs under high workload; they will grab hot-add memory whilst running under load and then never release it. This requires more careful setting-up and the SQLOS team have provided some guidelines for configuring SQL Server in virtual environments.

Whereas VMWare’s Memory Overcommit is well-proven in a number of different configurations, Hyper-V’s ‘Dynamic Memory’ is new. So far, the indications are that it will improve the business case for virtualizing and it is probably a far more intuitive technology for the average IT professional to grasp. It is certainly worth testing to see whether it works for you.

Fig 1: Beware the memory hogs, which are gross feeders on hot-add memory

Opportunity Nokia's

Andrew Clarke — Thu, 17 Feb 2011 15:25:00 GMT

Nokia’s alliance with Microsoft is likely to be good news for anyone using Microsoft technologies, and particularly for .NET developers. Before the announcement, the future wasn’t looking so bright for the ‘mobile’ version of Windows, Windows Phone. Microsoft currently has only 3.1% of the Smartphone market, even though it has been involved in it for longer than its main rivals. Windows Phone has now got the basics right, but that is hardly sufficient by itself to change its predicament significantly. With Nokia's help, it is possible.

Despite the promise of multi-tasking for third party apps, integration with Microsoft platforms such as Xbox and Office, direct integration of Twitter support, and the introduction of IE 9 “later this year”, there have been frustratingly few signs of urgency on Microsoft’s part in improving the Windows Phone product. Until this happens, there seems little prospect of reward for third-party developers brave enough to support the platform with applications. This is puzzling when one sees how well SQL Server and Microsoft’s other server technologies have thrived in recent years, under good leadership from a management that understands the technology. The same just hasn’t been true for some of the consumer products. In consequence, iPads and Android tablets have already exposed diehard Windows users, for the first time, to an alternative GUI for consumer Tablet PCs, and the comparisons aren’t always in Windows’ favour.

Nokia’s problem is obvious: Android’s meteoric rise. Android now has 33% of the worldwide market for smartphones, while the market share of Nokia’s Symbian has dropped from 44% to 31%. As details of the agreement emerge, it would seem that Nokia will bring a great deal of expertise, such as imaging and Nokia Maps, to Windows Phone that should make it more competitive. It is wrong to assume that Nokia’s decline will continue: the shock of Android’s sudden rise could be enough to sting them back to their previous form, and they have Microsoft’s huge resources and marketing clout to help them. For the sake of the whole Windows stack, I really hope the alliance succeeds.

RSS Feeds currently on Simple-Talk

Andrew Clarke — Mon, 14 Feb 2011 01:37:00 GMT

There are a number of news-feeds for the Simple-Talk site, but for some reason they are well hidden. Whilst we set about reorganizing them, I thought it would be a good idea to list some of the more important ones. The most important one for almost all purposes is the Homepage RSS feed which represents the blogs and articles that are placed on the homepage.

Main Site Feed representing the Homepage

..which is good for most purposes but won't always have all the blogs, or maybe it will occasionally miss an article. If you aren't interested in all the content, you can just use the RSS feeds that are more relevant to your interests. (We'll be increasing these categories soon)

if you want to get a more refined feed, then you can pick and choose from these feeds for each category so as to make up your custom news-feed

in the SQL section,

in .NET there are...

in Sysadmin there are

in opinion, there is...

in Books, there is

And all the blogs have got feeds. So although you can get all the blogs from here..

Main Blog Feed

You can get individual RSS feeds..

As well as these blogs, you also have the forums....

WebMatrix and SQL Server Compact Edition. Something for the rest of us?

Andrew Clarke — Fri, 23 Jul 2010 12:29:00 GMT

WebMatrix, currently in beta, is Microsoft’s newest initiative aimed at those of us who take no pleasure in using the overweight Visual Studio, but need to develop websites easily. For web-based applications such as blogs, forums, CMSs and wikis, WebMarix's design seems ideal, though it is currently working only in parts. At the heart of WebMatrix is IIS Express, SQL Server Compact Edition and Razor: The good, the bad, and the ugly. Someone is using SQL Server Compact edition? It seems so.

If you don’t know about Microsoft SQL Server CE, this was originally designed for SmartPhones. It is a small footprint, client-only, edition of SQL Server. It resembles most closely SQL Server 2000, simply because it hasn’t had much love from Microsoft, or the developer community, since then. It isn’t ideal for .NET work as it is an unmanaged database with managed wrappers. You can’t develop on SQL Server 2005 or 2008 and transfer your work across, because the last ten years of development, VARCHAR(MAX), stored procedures, Views, CLR Procs, CLR Triggers, Full Text Search, BLOB data, and so on, aren’t there. SSMS for 2005 didn’t support SQL CE at all, though SSMS 2008 does to a limited extent. It isn’t that easy to synchronise databases with kosher SQL Server. Up to now, it hasn’t been easy to deploy. However, it has good support for LINQ. I can see it being handy as a ’repository’ for Entity Framework, and there are some good reports from its use in this context.

Fortunately for WebMatrix, the Gallery of Applications, which provides a means of creating and deploying a wide range of well-known Web applications such as DotNetNuke, Wordpress and Drupal, mostly use MySQL and PHP, though some work with SQL Server Express. It is a curious experience having a Microsoft application installing MySQL and PHP on my laptop. I can imagine that SQL CE is probably there to sooth the ‘suits’ in Microsoft who would otherwise frown at the idea.

All in all, WebMatrix seems to be one of Microsoft’s better attempts to escape its geeky development culture, based on Visual Studio, to provide a simple tool for setting up web applications, but it highlights Microsoft’s neglect of the low-end database. We have and enterprise-scale monster in SQL Server 2008, but there is nothing to rival SQLite or Firebird for the developer who wants a small, embedded, easily deployed, client-side database, let alone poor VistaBD for being all this and .NET-based, managed, and reasonably compatible with SQL Server. So, are we, realistically, stuck with having to use MySQL for those applications that come under the radar of SQL Server 2008 Express? What do you use when you need a small-footprint, easily-deployable database, and you’re fortunate enough to be able to choose?

Winners of the Bejewelled Puzzle

Andrew Clarke — Mon, 03 Nov 2008 11:10:00 GMT

There are three winners:

Timothy Walters - for his second solution with the offset table; He gets a $50 Amazon Voucher.
Ryan Randall – for his first and second solutions; He gets a choice between a license for SQL Prompt and SQL Data Generator
DrLechter – for his first and second solutions; He gets a choice between a license for SQL Prompt and SQL Data Generator

Timothy Walters's solution is very simple and elegant. Almost all of the logic has been implemented in the lookup table and that makes the solution portable and flexible.

In addition, his solution can be easiely modified in order to show the directions of the moves by adding one more column to the offset table.

Here is how it can be done:


--====== Table matches needs to be loaded only once
CREATE TABLE matches(offsetRow1 INT, offsetCol1 INT, offsetRow2 INT, ofsetCol2 INT, directions VARCHAR(20))
-- for horizontal 
INSERT INTO matches VALUES(-1, -1, -1, -2, 'up')
INSERT INTO matches VALUES(-1, -1, -1, 1, 'up')
INSERT INTO matches VALUES(-1, 1, -1, 2, 'up')        
INSERT INTO matches VALUES( 1, -1, 1, -2, 'down')           
INSERT INTO matches VALUES( 1, -1, 1, 1, 'down')
INSERT INTO matches VALUES( 1, 1, 1, 2, 'down')       
INSERT INTO matches VALUES( 0, -2, 0, -3, 'left')     
INSERT INTO matches VALUES( 0, 2, 0, 3, 'right')            
-- for verical
INSERT INTO matches VALUES(-2, -1, -1, -1, 'left')
INSERT INTO matches VALUES(-1, -1, 1, -1, 'left')
INSERT INTO matches VALUES( 1, -1, 2, -1, 'left')
INSERT INTO matches VALUES(-2, 1, -1, 1, 'right')
INSERT INTO matches VALUES(-1, 1, 1, 1, 'right')
INSERT INTO matches VALUES( 1, 1, 2, 1, 'right')
INSERT INTO matches VALUES(-2, 0, -3, 0, 'up')
INSERT INTO matches VALUES( 2, 0, 3, 0, 'down')
 
--==================================================
;WITH CTE
      AS
  (
  SELECT
        [Row] = CAST( [#] AS INT ),
        [Col] = CAST( [Col] AS INT ),
        [Value]
    FROM bejeweled
        UNPIVOT ([Value] FOR [Col] IN ([1],[2],[3],[4],[5],[6],[7],[8],[9])) unpvt
  )
SELECT DISTINCT T.Row, T.Col, T.Value, directions
  FROM CTE T
      JOIN CTE T1
      ON T.Value = T1.Value
      JOIN CTE T2
      ON T.Value = T2.Value
      JOIN matches
      ON (T1.Row - T.Row) = offsetRow1
    AND (T1.Col - T.Col) = offsetCol1
    AND (T2.Row - T.Row) = offsetRow2
    AND (T2.Col - T.Col) = ofsetCol2
  ORDER BY 
 T.Row, T.Col

There is one more person – AmandaS that should be specially mentioned. Her second solution was very similar to Timothy Walters' and Ryan Randall's solutions, but was submitted five days later.

A few more people should be mentioned as well:

Saharafrog – interesting solution
Doug Stewart
Steve Munson - his solution shows the coordinates of the cells that should be moved and the coordinates of destination cells
RBarry Young
Dennis A
SkyBeaver

Litigation, and the 'Production' of Documents.

Andrew Clarke — Fri, 24 Oct 2008 11:59:00 GMT

Several recent events have been unsettling for many IT Managers. The recent collapse of banks has led to FBI investigations in the States, and groups of shareholders in the both the States and the UK have threatened litigation. At the time I write, there is evidence of criminal activity but not enough for prosecutions. There have also been price-fixing investigations by the UK's Office of Fair Trading.

All these investigations have required the 'production' of documents. I've taken part in several of these, and they can be extraordinarily frustrating and time-consuming, even when there is no question of a cover-up. I've also had to find data for MI5 as part of terrorist investigations. It is, of course, the duty of IT management to ensure that all electronically-held data is easy to find and conforms with the company's 'Retention Policy'.

Retention Policy?

In businesses, there is a huge amount of electronic data kept. This is mostly done in case of litigation, but corporate lawyers are generally reluctant to give clear guidance about this. When litigators seek 'production' of documents from an opponent, as they have every right to do, the confusion and duplication in document archives can make life very difficult for the legal department. There have been numerous court cases that have generated serious sanctions when legal counsel and their clients failed to produce documents during this 'discovery' process or worse, pretended that they had been destroyed, or had never existed.

'Discovery' is an expensive business, and the costs of litigation are enormously increased if lawyers cannot get hold of all the relevant documents.

It is not enough just to store everything. In fact, it can make matters worse because of duplication. Everything has to be read by someone who is aware of the legal ramifications of what has been written. To be able to cope, business must specify in their 'Retention Policy' what needs to be retained in terms of business risk, Business continuity, regulatory requirements, legal compliance, and statutory guidelines and requirements. These Policies must clearly state what data and documents can be discarded, and must be consistent across the enterprise.

There may be business needs that may exceed those requirements due to the potential historic, intrinsic or enduring value of the information. If the information has met all of these needs and is no longer considered to be valuable it must be disposed of in a way that protects privacy and confidentiality.

Any data can become the subject of a legal hold, which means that its retention must be indefinite until the legal hold is removed.

Emails cause unique problems because users rely on them for tracking projects and workflow. This makes any arbitrary 'disposal date' impossible. They also must be stored in such a way that they can be scrutinised. There can be no such thing as 'private emails' in the workplace. There have been several investigations where an email has proved to be the key evidence that has led to a criminal prosecution or the resignation of a politician

ILM Applications are worse than useless without a consistent policy for the retention and access of documents. Automating a confusion will just make the confusion faster. The policy has to be enforced and tested to make sure that, in an emergency, all necessary documents can be found.

I hope you never have to be involved in the litigation process. If you do, then the only way to cope is to have an effective policy, a way of disposing of all irrelevant or duplicate data, and the technology to search your data rapidly and effectively.

The LSE TradElect incident: We all lose

Andrew Clarke — Fri, 12 Sep 2008 05:46:00 GMT

The crash of the TradElect trading system on the London, and Johannesburg, Stock Exchange for seven hours was a disaster that probably led to the loss of around £3bn in trading.

The system was based on SQL Server 2000.

High volumes of trading on global exchanges were expected that day, following the news that Fannie Mae and Freddie Mac were being taken over. At 9:15 AM GMT, the Exchange's software failed due to "connectivity issues." It took Six-hours and 45-minutes for the London Exchange, along with the Johannesburg Stock Exchange, once more to start trading.

This incident has put a new question-mark on the LSE’s viability. In terms of cost, it is one of the most spectacular in recent history

The system was built, evidently, built using Windows Server 2003, SQL Server 2000, and a custom .NET application in C# by Microsoft and Accenture. Microsoft described the application as being ‘One hundred per cent reliable on high-volume trading days’ http://www.microsoft.com/uk/getthefacts/lse.mspx. The TradElec runs on more than a 100 HP ProLiant servers in several locations in London.

The system promised to give sub-ten millisecond response times, rather than 7 hour response times.

The LSE describe TradElect in their website as ‘…the Exchange’s new trading system. It brings unprecedented levels of performance, enhanced functionality and new services to our markets whilst maintaining our exemplary record for reliability.’

A spokesman told Reuters on Tuesday. "It was software-related, a coincidence, due to two processes we couldn't have foreseen, We've introduced a fix and we're confident it will not happen again.". She said the fault was not due to high trading volume. "That's a red herring. We were able to eliminate that pretty quickly."

So what was it? Another statement attributed the failure to network software issues between the LSE's Extranex private network (which links the exchange and clients) and the trading gateway to TradElect.. As yet, there is no confirmation of this by Cisco.

The whole incident has had the Linux world rubbing their hands with glee and maintaining that Microsoft technology just wasn’t up to it. Some commentators have even tried to make out that the LSE would have done better with a LAMP-based system with MsSQL.

The whole incident has been a major embarrassment for Microsoft since they made so much of the reliability of the system. Once the amusement of searching Microsoft sites for references to TradElect has waned, though, there comes the thought that, despite the crowing of the Linux gurus, it is quite likely that it had nothing whatsoever to do with the Microsoft technology, and it really was a problem with the software within the network. If this was the case, why hasn’t a statement been made? The delay is bad news because, in the meantime, what damages the reputation of TradElect , also damages confidence in the Windows Server, SQL Server , and NET platform. It was, after all, going to be the ‘benchmark’ application for the platform.

Pasting code into a Blog.

Andrew Clarke — Tue, 15 Jul 2008 07:45:00 GMT

One reason that I often hear for the apparent inability of certain people to blog is that there is no way for Community Server to display code. It isn't actually that hard. The obvious choice is to use Phil Factor's prettifier. It is the reason he wrote it. If you need to show exactly what is in Visual Studio, or SSMS, there is another way.

The problem you have is that Community Server is usually set to only allow FONT tags to enhance text. This makes life difficult, though it ensures that a blogger can't cause too much havoc with surrounding parts of the page.

Paste your code into Word. Make sure that it has pasted it in, using the 'keep source formatting' option. Then, save it as HTML. Alternatively, you can use Outlook. (I set the format of the message to be HTML. I then send the code to myself, right-click the results and select 'view source code'.)

I then take the HTML code into a nice text editor, 'top and tail' it (take out the HTML, BODY tags and the DIVs, leaving just the Ps in place) and do the following substitutions with the objective of taking out the SPANs (no need really as they don't get rendered anyway) and changing the Ps into BRs

FROM	TO	Result
style='font-size:11.0pt;font-family:"Courier New"'		Deleted all occurences
</p>\r\n<p class=MsoNormal	<BR	(this is RegEx) correct the spacing between lines. replace all paragraph tags with line break tabs

You may find that the font has been styled for a different size, depending on the setting in Visual Studio. This means that yo will need to remove style='font-size:10.0pt;font-family:"Courier New"' rather than 11.0pt.

Then create, or edit, the blog. Paste the modified code into the HTML pane (the tab is on the bottom-right of the panel in Community server) Select the code and, in the design pane, make the size of the code 2 rather than 3. Select the code and specify that it should be Courier

All that we're actually doing here is trying to remove all the Word silliness of trying to be all things to all men. All you actually need is the colour information and the non-breaking spaces to do the indentation. The final substitution makes the paragraph tag render as you want it to.

ALTER Function [dbo].[ufsRemoveDelimited]
    (
      @String VARCHAR(MAX),
      @OpeningDelimiter CHAR(1),
      @ClosingDelimiter CHAR(1)
    )
RETURNS VARCHAR(8000)
AS BEGIN
    DECLARE @newString VARCHAR(8000)
    IF @OpeningDelimiter = @ClosingDelimiter
        BEGIN
            RETURN NULL
        END
    IF @OpeningDelimiter + @ClosingDelimiter + @String IS NULL
        BEGIN
            RETURN @String
        END
    SELECT @NewString = ''
    SELECT @newString = @newString + SUBSTRING(@String, number, 1)
    FROM    numbers
    WHERE   number <= LEN(REPLACE(@string, ' ', '|'))
            AND CHARINDEX(@OpeningDelimiter, @string + @OpeningDelimiter,
                          number) < CHARINDEX(@ClosingDelimiter,
                                              @string + ' '
                                              + @closingDelimiter, number)
            AND number <> CHARINDEX(@OpeningDelimiter, @string, number)
    RETURN @NewString
   END

Here is some C#, copied out of Visual Studio 2005

// Marshal.cs
using System;
using System.Runtime.InteropServices;

class PlatformInvokeTest
{
    [DllImport("msvcrt.dll")]
    public static extern int puts(
        [MarshalAs(UnmanagedType.LPStr)]
        string m);
    [DllImport("msvcrt.dll")]
    internal static extern int _flushall();

    public static void Main()
    {
        puts("Hello World!");
        _flushall();
    }
}

Here is a small sample of VB code

Private Sub Button1_Click(ByVal sender As System.Object, _
    ByVal e As System.EventArgs) Handles Button1.Click

    ' Stores the return value.
    Dim RetVal As Integer
    RetVal = MBox(0, "Declare DLL Test", "Windows API MessageBox", _
        MB_ICONQUESTION Or MB_YESNO)

    ' Check the return value.
    If RetVal = IDYES Then
        MsgBox("You chose Yes")
    Else
        MsgBox("You chose No")
    End If
End Sub

A Prompt Delivery

Andrew Clarke — Wed, 02 Apr 2008 02:57:00 GMT

I wandered into Red-Gate's offices the other day. I'd been momentarily distracted by the autumn leaves, thoughtfully strewn in the hallway, probably by a visiting Zen Buddhist to remind us of the sacred presence of nature even in the Techno Park. Suddenly I was waylaid by the charming Tanya to take part in a bug-hunt. Somehow, I found myself in a room full of laptop computers and various developers, varying from the apprehensive (SQL Prompt developers) to the smug (ex-SQL Prompt developers).

We were giving SQL Prompt 3.8 RC a quick post-test check-over before the new release candidate version of 3.8 is made available.

The interesting thing about SQL Prompt 3.8 is that it is now using the code reformatter from SQL Refactor to lay out your SQL code nicely. This means that, for the first time, even the users of Query Analyser now are able to reformat code automatically. There are twenty different aspects of the reformatting you can change, in four different sections. It is a very clever piece of code that, for me, doubles the usefulness of SQL Prompt.

There are one or two other features that are less immediately exciting than the reformatting code but would make a big difference to me. One of these is the 'tooltips'-style help for whatever object that you rest the mouse pointer over. Wonderful. I'm an old-stager with IDEs. I like a nice clean window free from clutter, and loathe having to do incessant clicking in the Object Explorer to do stuff. This is why I reach for such old-time practices such as sp_help and sp_helptext. If SQL Prompt can help me keep the clutter away so I can remind myself of the parameters to a routine or the columns of a table without having to scrub what's in the results window, then that's a real bonus. Other new features include Object Definitions, and a new Summary view in the schema panel.

SQL Prompt is one of those product that one can love, but never unconditionally. There are so many different ways that Database Developers work and it is presumptuous to think that it could ever be possible to construct a tool that would satisfy everyone. Perhaps that is why Microsoft have taken so long to provide intellisense for SQL Server developers. It has to work for such a diverse community of users. However, by diversifying into managing layout, and providing all these nice features for even those unfortunates still wrestling with SQL Server 2000, SQL Prompt's future is assured, whatever arrives by way of intellisense in SQL Server 2008.

I found SQL Prompt irritating to use until I adopted the obvious strategy of setting the triggering to manual (this should have been the default). Now you just invoke SQL Prompt just at the point of those senior moments when you forget the name of the table, routine or parameter. Another handy feature is being able to limit the trigger words that invoke SQL Prompt. There is a bit of work to do in order to make it function the way you want but the effort is worth it.

The Red-Gate bug-hunts are good fun for the huunters, but painful for the application developers whose work is being treated the same way as a fox by foxhunters. TallyHo! I sat next to a developer who had the habit of throwing his head back and laughing in an evil gloating way every time he found a bug. I gazed in wonderment at the restraint shown by SQL Prompt's developers whose welcoming smiles only glazed over momentarily in response to the baying pack of bug-hunters, as they pushed firmly into the back of their minds any homicidal instincts towards the gleeful bug-hunters 'on the scent'. There is a certain etiquette in conveying the news to a hard-working developer that there is a tiny flaw in his masterpiece. Perhaps the pretence of amazed surprise that such a thing could possibly exist. Certainly, gloating or reproach is not an option, especially if, one day, the roles will be reversed, and it is your software being torn to shreds.

Fortunately, the software worked surprisingly well for a beta (it is now a release candidate as I write - 2nd April 2008) so we're getting it ahead of schedule. I, for one, will be pestering the developers over what I see as its little quirks and irritations, but I'm fully expecting people on The Red-Gate Product Forumpraising the very features I find irritating. That's all part of the software development process, I suppose

Coram magnificos manifeste scalpere nolis ...

Andrew Clarke — Wed, 27 Feb 2008 04:56:00 GMT

'Coram magnificos manifeste scalpere nolis
Torquendo digitos nares…'

Daniellis Becclesiensis c1200 AD

(meaning: Do not pick your nose in front of managers)

One of the delights of being Simple-Talk’s subeditor, under the watchful eye of Tony (sir), Anna and Simon, is to interact with our team of regular writers. When I use the word ‘Team’, I do so rather loosely, as writers come and go under the influences of the pressures of other work, or life events. By a strange coincidence we see the return of two regulars to the team after a gap of two years.

This issue welcomes back Amirthalingam Prasanna. Prasanna last wrote for us two years ago, and his five articles, with their clear tutorial style, have always been popular. He has returned with an update of his .NET Collection Management 08 December 2005 , called .NET Collection Management with C# We’d particularly like to cover Entity Framework on Simple-Talk, and we’re hoping that Prasanna will continue his output with a Cribsheet on the subject.

Richard Morris, being a professional journalist, does not suffer from the need for pencil-chewing and inspiration that affects most of our authors. He needs no encouragement: his articles are there, on time, every time, and are always stimulating to read. He writes best when he feels a genuine indignation about the subjects of his investigation, and in this article, where he investigates the troubling fact that almost all the British government IT contracts go to a handful of enormous multinational IT companies, the heat is palpable.

Phil Factor is not the most malleable of authors. Articles arrive with a quiet apology like ‘I know you wanted an article on maintenance plans, but this one just popped out.’ The article will be a mock pantomime, an elegy on indexes written in the style of Tennyson, or a treatise on the difficulties of running a project, written in the style of GK Chesterton. I have before me Phil’s translation of Machiavelli, rewritten very slightly and chillingly contemporary. Also Phil’s translation of the mediaeval (twelfth century) latin poem ‘Urbanus Magnus’, by Daniellis Becclesiensis, phrased for the modern office-workers, from which I have taken the quote that starts this Blog.. It means ‘Do not pick your nose in front of managers’. (literally ‘In front of grandees, do not openly evacuate your nostril by twisting your finger’). Other advice from the great work includes ‘if the wife of your manager attempts to seduce you, feign illness, as this is a no-win predicament for you.’. It is the first book of etiquette ever written in Western Europe.

Under Robyn’s restraining influence, Phil’s work is more sober, and more consistent. Robyn is now on maternity leave but we are very fortunate that she continues to produce her articles for us, somehow taming Phil’s creative spirit. ‘The Alerts, Soup to Nuts’ is their first attempt at a ‘Soup to Nuts’ article. It all seemed so easy at first, evidently, but the more they got into the subject, ran tests and so on, the more they realised how little they understood of the subject. A quick rustle through the SQL Server Administration books convinced them that few others did either. The whole subject of tokens seemed to be completely neglected in many of the books on SQL Server Administration. After a long struggle, I think that the team succeeded well.

Arthur Fuller is a stalwart ‘Friend of Red-Gate’ whose first article for us was over two years ago. The Database from Hell is his fifth. This was supposed to be the first draft, but we liked its freshness so much we popped it in without the usual tidying and editing. We apologise in advance to the irate Scotsman who dislikes it when a Red-Gate product get a mention, but it was all part of the story. The illustration we used was adapted from Eric Liberge’s ‘Tonnere Rampant’, with Eric’s permission, and we are delighted with it. He is a great illustrator, whose work deserves to be more widely known.

The Simple-Talk newsletter continues to demand more attention than it should, because it seems to be unreasonably singled out by some versions of Microsoft Outlook as spam. Conspiracy theorists might have a field day with this, but it is very likely an over-zealous algorithm. We’ll be experimenting with the layout in the next few weeks and any opinions you have will be gratefully received.

Thick and Fast

Andrew Clarke — Thu, 07 Feb 2008 07:37:00 GMT

There is a great deal of activity on the Red-Gate forums at the moment. In fact, they are a fascinating read. Often, the threads take off in ways that nobody can really predict. My particular pleasure is in reading the 'off the cuff' remarks such as 'I have servers in NYC, Los Angeles, Pakistan, and India.' or 'I imported roughly 200 servers with over a thousand databases. The servers are distributed throughout the mid-West', or 'The number of servers I will be supporting is jumping to over 200 in the next few months' that give a glimpse of life at the DBA coal-face.

One of the best, if you haven't looked, is the thread that started off on the Refactor Forum. Andras asked 'We support nearly forty options to customize laying out SQL, but is it enough?'. There were 41 responses, and the resulting thread was wildly popular. There are so many different ways of formatting SQL and there seemed to be an advocate for every one.

Here is a subject where programmers will argue for hours. For example, although I agree with the great Joe Celco (SQL Programming Style 2005) about putting reserved words in Upper Case, we had people wanting everything in lower case. You're never going to please everybody, especially not Joe.

So, is there going to be a version that includes all the different layouts? Mike O'Neil wasn't the only one wanting to know. I'm a great fan of Refactor, but I've never quite tweaked the layout exactly how I want. Wthere wasn't great encouragement about a new version despite the interest in formatting. The roadmap only goes to the end of Q1 of this year so lets' hope Q2 for a new version as this is the Red-Gate tool I rely on the most.

There is hope, though as SQL Prompt 3.8 will hopefully be available at the end of Q1. The Pro version evidently incorporates the Layout SQL functionality from SQL Refactor. It is an interesting move, since a lot of people aren't quite sure of the difference between the two products. and one wonders if the new Pro version will prove to be the logical line of development for the products

SQL Prompt 3.7 patch release came out quietly, but the product is improving the whole time. I very quickly learned to set 'triggering' to manual (Cntl+Spacebar) when using SQL Prompt, and find it very handy as it cuts down on the need to keep accessing the Object Browser.

The SQL Data compare forum had a wonderful thread where people were asking for enhancements to the product and Richard kept bobbing up and explaining that they were there already. For me it was rivetting to hear of extreme problems encountered by users such as 'tons of tables with no indexes'.

Over on the Alpha/Beta forums, the buzz around the Data Generator is dying down now as the developers start to assimilate all the feedback, and code the good ideas into the next beta. Now it is the turn of Dan and the SQL Response Beta.

As I write this, it is early days, and I suspect that there will be a huge diversity of opinion on the things that should be monitored in SQL Server to give it a health-check. In the pub, it is always a debate that ends in flushed faces and Beer-glasses getting thrown.

The Red-Gate site now has Blogs. I must admit that I had slightly mixed feelings about this, probably because someone tried to copy the existing blogs across from Simple-Talk and all the comments dropped off like autumn leaves, causing much confusion. However, Neil rallied round and started cross-posting from his wonderful blog on BusinessOfSoftware, However, the true purpose of the Red-Gate Blogs was illustrated best by David Connell in How to write a Generator for SQL Data Generator 1.0 Beta, whereas over here, Lionel showed how well a Red-Gate developer could blog on Simple-Talk with his SQL Puzzle 9