ITGumbo: spicing IT up

As a precursor to OSCON this week, Forrester stepped forward with a new report titled  "Market Update: Open Source Databases," one that tags the current market for open source database market at $850 million, with growth to $1.2 billion by 2010. I haven't seen the entire report but I did get around to reading the excerpt available on Forrester's web site. I'd sum up my thoughts on the future of open source databases by saying that they will continue as a tool of choice for applications that are ants in comparisons to the sledgehammers of the database world (Oracle, IBM). Obviously, Sun's strategic vision for MySQL will play no small part in this picture. However, open source databases will fuel a great deal of the innovation that has yet to surface (think cloud computing, corporate Web 2.0 and improved forms of content management).

New types of applications that have yet to breach mainstream are already being developed atop open source databases. We saw this with MySQL and web applications and we'll continue to see a similar relationship develop between open source databases and application developers who sit closer to the cutting edge. Additionally, I'm looking for more and more consulting houses to begin offering implementation services for open source databases as the demand for newer types of solutions emerges. Still, it will remain difficult to determine just how big the pond is for open source databases since they will continue to power a great deal of applications in anonymity.

I also fully expect that open source database vendors will begin to better harmonize the potential of their communities with business models. As it stands, the open source community is the place to go for support with non-production deployments. And there are more effective ways to align this with the delivery of 24x7 premium support. Tiered layers of support patrons could emerge as a monetization vehicle that serves user who don't want to sign up for the traditional support contract but want a fast track for feedback to questions and issue resolution.

I've got the feeling that we'll also witness the emergence of Ingres as a more noted player in this segment. In the past, it has lacked the visibility of a MySQL or PostgreSQL/EnterpriseDB. However, since splitting from CA two years back, it has continued to build out a comprehensive set of database features. Still, the company needs to figure out how to gain back the implicit sense of credibility that being associated with CA brought it amongst the enterprise crowd.

Along the same lines I see Firebird retaining its occult-esque status with a subsection of developers who recognize its value as solid technology. At the current moment, Firebird lacks a cohesive face and/or supporting vendor so it continues to fly under the radar despite its relative strengths. It won't be able to establish enough penetration outside the developer community until the prospect of fully supported production deployments becomes a reality. Then again, I'm not so sure that's the ultimate goal of the project.

Now that MySQL has decided to ready the final Release Candidate (RC) for version 5.1 instead of moving forward with a General Availability (GA) release and Michael Widenius has extended an bug filing call to its user community. I've been thinking about where MySQL stands as an official 5.1 release has been pushed back. For all it's dominance as the glue for LAMP stacks and engine for Web 2.0 infrastructure, my thoughts on MySQL have little to do with Sun's $1B acquisition or whether it can topple Oracle DB. Rather, they're focused on core realities associated with the product itself. Namely:

• For those who remember, MySQL 4 left a bad taste in the mouth of more than a few. Not that anyone expected it to be perfect but the lack of so called "advanced" features like triggers, sprocs, udf's were a turnoff. To this day there is a healthy segment of PostgreSQL users that swear MySQL is an amoeba in comparison, and most base this assertion on having used MySQL 4...seriously. A lot of this is understandable, version 4 lacked support for things like nested SELECT's so it's to be expected that some folks' experience was less than ideal. Unfortunately, those who fled MySQL before the 5.0 was available missed out on some notable additions, including syntax changes that moved it closer to ANSI standards.

• The 5.1 GA will be the first widely available release with native XML functions. I'm waiting to see if there's any indexing support behind all the xPath.

• MySQL has consistently trumped PostgreSQL with it's ease of install, maintenance, and lower levels of complexity. Still, MySQL lacks what some would call the basic ability to store a timestamps with greater than a second accuracy. In the operational world a database that can't do that would be considered a step away from Access. At least milliseconds are required and microseconds are preferred. Note: MySQL currently recommends stripping the sub-second information from the time sent, storing it in another column, then implementing a view which combines them back. However, for functionality that classed as basic this comes across as overkill...especially when the database's calling card is simplicity.

• Apparently, before being bought by Sun, MySQL AB were already contemplating making the move to more proprietary tools. From what I've been told, it was agreed that this was a natural progression that would enable them to monetize pieces of the pie. Talks were set in motion and the decision was left on the table...then Sun purchased them. After the acquisition, Sun didn't see the need to/benefit in proceeding down that path in the near term but took heat as some fingered them as the original source of the idea.

At the end of the day, the delay of version 5.1 GA will afford us with a look at what the priorities for Sun's adopted open source child are at this point. The state in which 5.1 hits the ground running will reflect how Sun intends to position MySQL moving forward. My hunch is that we'll witness longer gaps between releases as more focus is paid to QA, testing, certification and feature stability than in the past.

Lately, facing the prospect of picking up blogging after some significant time away from it, I've been pondering the topic of inertia. Most are already familiar with the term, but underestimate how this fundamental principle is applicable outside classical physics...even when examining the topic of attitudes towards open source in the year 2008.

For those who've witnessed the growth of open source from fringe movement to full-fledged force, it goes unnoticed that most decision makers continue to harbor woefully outdated ideas and perceptions about open source software. Part of this might sound absurd at first, so to better illustrate my point here are some categories of interest as my experiences have revealed to me:

1. Totally unaware - Have never heard of open source (a fringe minority).
2. Not interested - Have heard but not motivated to even consider it.
3. Interested but inactively so - Might read, discuss or even inquire but nothing further.
4. Leaning towards exploring - Actively informed and interested but still several steps away from a pilot.
5. Actively exploring - Already piloting.
6. Adopters - Been there done that.

I suffices to say that categories 2 and 3 claim the vast majority of the enterprise crowd. Part of it is that within the modern enterprise (especially larger ones) higher degree novelty = higher degree of risk. So while time might be the absolute test for all things (over the long haul) it's still safer to fall within accepted boundaries (in the short term). Meaning open source isn't just competing on grounds of functionality, price and "value" but also against organizational inertia. The patterns, habits and assumptions built atop years of playing in closed gardens is a strong force, to say the least.  Even as open source has proven itself "good enough" where it matters most, these ingrained habits and viewpoints represent another hurdle.

To overcome this, open source must represent enough gain to soften widely-held doubts about the model itself. Once again, these concerns might sound a bit 2003-ish but are still relevant today in 2008.

1. Security of the software - The association with open code bases and vulnerability still exists.
2. Availability of service and support - A great deal of open source packages don't have a supporting organization behind it...nor should they.
3. TCO - Not just monetary costs but those resource and knowledge based.
4. Legal issues - Self explanatory
5. Viability of the open source communities - Can communities be counted on to deliver consistently? Will open, collaborative software development model stand the test of time?
6. Complexity of adoption/learning curve - What exactly does it take to establish tailored policies, governance, etc.?
7. Product maturity - The one concern that will gradually dwindle based solely on the passing of time.

Where the above have been shown to be addressed, open source has flourished (commercially and otherwise). Where one or more of these doubts linger, growth has been substantially slower and/or sporadic. Unfortunately, the inactive passing of time during which open source "sticks around" does not guarantee that the above will be proven moot. Neither does the financial viability of individual open source vendors. It will take a concerted effort to continue to push the open source model with the objective of chipping away at some of the misplaced concerns that linger.

In my last post I attempted to overlap what it would take to establish a middle ground and the concept of open source driven solutions.  I'm not sure how well my point went across, but I am certain that the notion of a new wave of solution that not only includes, but is rooted in openness is one that's here to stay. With that in mind, I think it's relevant to look into what it will take to facilitate the shift towards flexible, truly open solutions. More than anything I think this entails a change in perception regarding what/how sizable a role open source will play in the mostly nebulous world of "solutioning." In effect, what needs to happen in order to kick-start the evolution of open solutions from embryonic towards maturity?

To those allergic to buzzwords, the term solution might incite a sneeze. However, upon further examination a great deal of technology is delivered as a solution, that is a bundled, ready-to-use component. The consumer technology market, by itself, is rife with examples (see: the iPod, most home PCs). And while the concept that consumers want to purchase products that fit a specific niche practically out-of-the-box, the notion that the same holds for large companies isn't as evident. To be fair, it's not possible to mass produce ready-made technology for organizations with narrowly defined business requirements and use-case scenarios. And in the case of open source, there are hardcore realities that stand in the way of it playing a larger role in driving greater time-to-value and cost efficiency:

• Integrated designs. One solution size does not fit all and the fit is typically determined by the industry in question. This implies a knowledge of specific segments drawn out in the form of industry solution maps.
• The importance of the sales cycle. Solutions are sold, first and foremost. Open source is still being stealthily transported into a great deal of enterprise environments with the sales process a shortened after thought. If the notion of open source solutions is to take hold, it must be sold and accepted wholeheartedly not smuggled in and permitted to stay afterwards.
• Overlap with industry proven models, best practices and methodologies is key. Acronyms like CBM, IFW and BPM might not be everyday terms but they are industry proven and widely accepted as the basis for solutioning.
• The criticality of a top-down approach that initially prizes business value over technology. I won't pose the bottom-up mentality that has buoyed the open source software movement for some time now, against the seemingly inherent top-down nature of most hierarchies. However, it pays to note that technology selection/procurement is at its core a business decision. And unlike individual products, integrated solutions speak more to the business side of things than anything else.

Finally, if open source solutions are to evolve they must rely on more than being cheaper. Especially since the marketplace tends to base its decision on more factors than just price alone. The market for solutions is driven by participants who exhibit demand for addressing their industry-specific performance drivers, are modular and customizable. Add to the fact that providers which can accelerate time to value, reduce risk and deliver integration are its leaders and it becomes evident that the road to broadly relevant open source solutions is a tough one, indeed.

About the blogger: Alex Fletcher is lead industry analyst at Entiva Group Incorporated, a research and analyst firm which specializes exclusively on the open source software industry. In addition to his analyst coverage activities, he advises organizations of all sizes on establishing governance, strategy and policy surrounding use of open source software as a competitive differentiator. Alex has prior experience as a consultant, software engineer and start-up founder. He can be reached at alex dot fletcher -at- entivagroup dot com.

"What would you add - if anything - about using organizations like the Eclipse Foundation, the ASF, or other standards/code bodies to have a sort of neutral-ground and/or organization "middleware" for all this?"

While the concept of middleware that sits between an increasing array of quality software, produced as output of the open source development model, sounds abstract the growth in demand for open source has created a void for just that. This middle ground is essentially a nether region between software vendors/system integrators and the businesses looking for last mile solutions. Typically, for the larger companies and organizations it's players like IBM GBS and Accenture Global Services that go that last mile to deliver the notion of an integrated solution (excuse my marketing-speak). However, this is done taking a proprietary approach (products, tools and processes). Which proves wildly profitable for both of the aforementioned companies, still what has yet to be seen is who will play the role of coagulating some of the disparate sources of open source, refining it and delivering as consumable pieces that meet a specific business need.

I'm a big proponent that software is bought for what it does instead of what it is. High-end ERP implementations don't ring up 7-8 figures on account of simply being an ERP rollout, rather it's the fact that these systems are often the lifeblood of modern business operations. And there's a growing role for open source within the context of ever narrowing market segments. The Eclipse Foundation has caught wind of this reality and begun to branch out in the direction of specialization. And they're not alone, the OSA has made headway towards giving life to the open solution. These efforts will be instrumental in making open source feasible for segments such as the SMBs, microverticals, service industry verticals and emerging economies.

This is precisely where the middleware analogy is applicable. In the case of organizations like Eclipse and the Apache Software Foundation they are the glue/packaging that enable that sits between the raw productive power of the open source software model and the industry-specific needs of its consumers. Strangely enough, the need for a middle ground goes almost entirely unmentioned. Yet in the absence of an established group of open source behemoths that influence macro-level trends increased prevalence of organizations that serve as a neutral buffer is a positive. Realistically, these organizations don't even have to resemble Eclipse or Apache. And I fully expect that more OSA-style groups will sprout, as profit-turning entities like SIs or vendors begin to realize that it works to their benefit to cooperate towards delivering a collectively neutral approach to "solutioning."

Whether this middleware is spurred by marketplace demand or the proactive foresight of players in industry has yet to be seen. Additionally, increased consolidation amongst open source participants might bring about de-facto platforms fit with ready-made ecosystems that can standalone from one given end to another. However, to wait for impending consolidation is reactionary and strategically sloppy, at best. Especially since in the light of every acquisition (large and small) there is considerable room to pursue the middle ground.