Glasnost in IT: Discarding the Old Certainties

Three of Redgate’s tools are now part of the 2017 release of Visual Studio Enterprise (as announced last week), as components of the Data Storage and Processing ‘workload’. Some might be surprised to find third-party tools being included in the Visual Studio installer, but it’s an interesting sign of the times. No longer can any single vendor hope to achieve market dominance in the way that IBM, SUN, Microsoft and Apple once managed in the last century.

The old certainties are long gone. There is a much wider appreciation of the contribution of the third-party and open source ecosystem around a product. Users of IT systems now expect their platforms, tools, frameworks and applications to work together cooperatively, to common standards. To be competitive, organizations need to be able to exploit all the opportunities presented by Cloud services, tablets, phones, devices and network services. They need to be able to connect and monitor a range of systems, whether they are on Windows, Linux, iOS or MacOS.

Those who have grown up in a more divided industry must be astonished to see the metaphorical “berlin wall” come tumbling down. Microsoft have become a major open-source contributor to projects such as OpenSSH, FreeBSD, Mesos and Docker. SQL Server now runs on Linux, Bash is there in Windows 10; the ASP.NET Core framework is open-source, built around NuGet packages, and the apps will run on Windows, Mac and Linux.

Suddenly, thousands of free, command line compilers, tools, utilities for system monitoring and analysis, are available for use. If you need something, it’s probably there; but are we ill-prepared and ill-quipped to exploit the new freedoms this may bring?

On Simple Talk, our readers are increasingly working in a multi-platform, multi-cloud, multi-database, multi-OS industry where everything must work together. This will, inevitably, be reflected in changes in the industry, and they aren’t necessarily technological changes. We need DevOps and ALM/DLM to help us get cleverer in the way that we integrate, test and release applications. We also need to be a great deal more savvy about the way we do our architectural planning. Simple Talk content has changed over recent years to reflect this.

When applications were created specifically for either Windows, Apple or Linux boxes, we could just design applications the way that the platform mandated. In this new heterogeneous IT world, there are many more new, wonderful and exciting ways in which we can completely fail to deliver applications.

Commentary Competition

Enjoyed the topic? Have a relevant anecdote? Disagree with the author? Leave your two cents on this post in the comments below, and our favourite response will win a $50 Amazon gift card. The competition closes two weeks from the date of publication, and the winner will be announced in the next Simple Talk newsletter.


  • Rate
    [Total: 2    Average: 5/5]
  • Peter Schott

    I’ll admit I was surprised to see a free version of SQL Prompt included in VS 2017 Enterprise. (Pleasantly surprised, but still surprised.) It’s definitely an odd world when you can show people MS code running on Linux and Mac when there was little or no support for those platforms before. Even their mobile “strategy” seems to be to get their apps onto iOS and Android before their own platform – even w/ UWP and Xamarin in the picture.

    I’ve appreciated the shift lately to include more DevOps and DLM in the articles. Database change management has been long neglected and while some solutions have been around for a while, there are indeed a lot of new ones coming out all the time. It’s no longer possible to really keep up with all of the changes in one product. That’s now specialized quite a bit.

  • Jeff Moden

    From the article:
    “Suddenly, thousands of free, command line compilers, tools, utilities for system monitoring and analysis, are available for use. If you need something, it’s probably there; but are we ill-prepared and ill-quipped to exploit the new freedoms this may bring?”

    Heh… I’ve been through the beginnings of all this. I was working for a rather well known media company. It was way back when 2005 was only a year or so old and the company I was working for was still using SQL Server 2000 and DTS. They downloaded dozens and dozens of files each day. The files were CSVs that were meant to be loaded into spreadsheets. They were “horizontally oriented” as most people would expect a spreadsheet to be instead of “table-like”. The DTS packages were designed to get each file ready to load into real, normalized tables. The files were typically only 30-40 thousand rows but had a varying number of pairs and quads of related columns (up to about 800 columns) and the first 9 columns made up what could be used as a Primary Key for each row.

    It took the DTS package 45 minutes to get just one of the files ready for import, never mind actually importing it. 45 MINUTES FOR ONE FILE and we got dozens and dozens each day. There wasn’t actually enough time in the day for the “ETL” server to get all of the files ready for import. They couldn’t fix it, either, because the people that wrote the parts of it all had all left. There were Active-X scripts, Perl Scripts, VB scripts, DLLs built by products no one knew how to use and, besides, they couldn’t find the source code anymore, and a couple of other scripts that I had no clue as to what they were written in.

    It was a “Tower of Babel” and I’m afraid for the future of companies that can’t settle on a technology and then hire people to use that technology. It’s as bad or worse than a lot of the forum posts that have reference UNCs in them that no longer lead to anything but a 404 because the site went away.

    I fixed the code. I wasn’t able to fix the code because I was some sort of multi-lingual computer genius… it was because I was a mostly single-lingual programmer that could kind of figure out what they were trying to do in the scripts and then make a good guess about what the missing source code may have been doing. I wrote it all in the language of the database, T-SQL, using some old “friends” including the Tally Table, a Tally Table based splitter, and BULK INSERT. When I got done, and it only took a week including full unit and integration testing, it was not only getting 8 files ready for import in 2 minutes total, but it also did the actual imports with the proper DRI in full swing.

    From the article:
    “The old certainties are long gone”.

    Actually, they’re not. The old certainties prevail. People will spend hour upon hour learning the “thousands of free” too-cool-for-school latest and greatest end-all-to-be-all shiny new objects only to have them suddenly disappear because the 3rd party went under, the web site disappeared, they don’t work in the latest version of SQL Server, or they’re no longer too-cool. All that’s a certainty because it repeats the same pattern month-after-month, year-after-year, and decade-after-decade. The really bad part is that when these flash-in-the-pan tools do go up in smoke, they leave companies hanging just as sure as the employees that brought the tools in to a company and then left. Worse yet, you end up with things that take 45 minutes to run one thing instead of 8 things taking just two minutes.

    I don’t know how EDI, Java, XML, and JSON made it through the gambit but I guess something had to. 😉

    That old code I wrote still works even after more than a decade of SQL Server upgrades although it has gotten much faster thanks to a dozen years of improvement in the hardware.

    And that’s the other certainty. Hardware can change. SQL Server can change, Where the hardware may physically live can change. How you communicate with the hardware can change. Data can get bigger and wilder in format or, lately, lack of format. But the one thing that doesn’t change is the basics that all of those other changes still rely on.

    Based on what I’ve seen, especially on the database forums, I’m certain that a whole lot of people are missing the basics when it comes to relational and other types of databases. They don’t spend any time at it because they think it’s not new and it’s not cool.

    If only they knew… the basics.

  • whipdancer

    Some of the changes are welcome. I specifically look forward to the concepts that docker/containers bring to the table. They represent a paradigm change (with respect to app dev/deploy) from the more traditional server based mindset.

  • Keith Rowley

    This makes a ton of sense to me. No one person can do everything, and we have long since reached a point where no one company can do everything. I must say, while this seems new I would note that Microsoft has allowed third party programs to run on windows since at least the 3.1 days. (That’s as far back as I have any experience to go on.)

    What’s new is Microsoft including third party products in their distributions and I must say this is really cool. Congratulations by the way.