The Great Database Migration to Open Source – A “How-to” Guide for the Pilgrimage
Ed Boyajian, CEO I EnterpriseDB
Database migrations from closed source to open source have become a particularly hot topic recently as I have been asked by customers, analysts, investors (and a few competitors!) what’s driving the accelerated changes in enterprise and government IT departments.
We have been both pursuing and pushing this change to Postgres for over 10 years now, so migration is not new to us at EDB (we and our customers have long thought it was “hot” to displace Oracle and the big DB players with open source) but what struck me in the nature of the questions I now get is how misunderstood the market of database migration is. And yes, it is a market. So why is this market so misunderstood?
Head Nod to New Apps – That’s Where the Honey Is
Let’s face it, we are all juiced by creating something new. That’s as true in IT today as anywhere and for good reason. Business and corresponding IT development models continue to go through a big transformation and the high demands for revenue growth demand equally high growth in the IT support they get. Gartner talks about the New Digital Business Model. In Digital Business Key Initiative Overview by Jorge Lopez published 21 July 2015, Gartner states, “Digital business is the creation of new business designs by blurring the digital and physical worlds. An unprecedented convergence of people, business and things will disrupt existing business models. Businesses will introduce new products and services, even new business models, which connect people, business and things (physical objects that are active players and contribute to business value). This will drive revenue and efficiency that will ensure businesses’ ability to compete for years into the future.”
A focus on new, incremental opportunities driven by technology understandably draws the significant market attention away from the topic of migration. That’s acutely true in data management these days. By my count there are at least 4 upstart software company “unicorns” (> $1B valuation) that are database players: Cloudera, Hortonworks, MapR (all three Hadoop plays) and MongoDB (document database). Between them they have garnered more than $1B in investment capital. All are chasing new workloads and add a new growth node to the $36B database market. How big that growth node will be is still a massive open question (all together this group’s revenues are still well below $1B or <3% of the market).
I get it. I like this market too. A lot. Look below those unicorns and you see others including my company, EDB with Postgres, and then there’s DataStax with Cassandra and MariaDB with MySQL also getting bites at the new workloads. At EDB, I estimate that roughly 30-40% of all our business today comes from customers who are writing new apps and have strategically selected Postgres as the new go-to open source database. EDB customers developing new applications of engagement include: Clear Capital, WholeWorldBand, KT, Coresystems, and InMobi.
But what about the other 60-70%? For EDB, that’s the other end of transformation and change. And that’s where the migration happens.
Database Migration – That’s Where the Money Is
The math on this is pretty simple. Let’s look at the big picture. The database market is about $36B in 2015. Gartner and others estimate it grows at about 8% per year. That means the market adds about $3B per year. For the sake of argument let’s call all that $3B the “new” discussed above. That leaves a pretty big “migration” market – $30B+. That also points to the magnitude of the opportunity for savings within the enterprise
We have hundreds of customers who are on the strategic migration journey, including over 80 of the F500. The economics can’t be ignored.
The transformation that is the “migration market” is happening fast and at a large scale. We see it happening predominantly on premises where the majority of enterprise applications run. Our biggest customers stack up something like this:
- G1000 Telco – Using EDB Postgres as the new standard DBMS (not just open source DBMS) systematically replacing Oracle on hundreds of applications and saving eight figures a year
- Major US insurance provider serving 10’s of millions of customers with EDB Postgres Plus
- Leading global financial institution is aggressively migrating applications from Oracle to EDB Postgres across its organization
- One of the world’s largest stock exchanges by market capitalization has standardized on EDB Postgres for all new applications
For these companies, the common threads in their migration journey are:
- C-level drive to change the economics of IT and use money overspent in DBMS to fund other strategic business and growth initiatives
- Recognition of the massive overspending on their proprietary database vendors (mostly Oracle) and an impetus to change it (pricing and contract pain with Oracle is the single most common catalyst)
- Early POC and proof that open source alternative like EDB Postgres meet the technical and business requirements
One key question I get is, “what about cloud…is that driving the migration market?” Amazon just entered the migration scene with a service to support their public hosted, managed service segment. We have also been serving this segment with AWS since 2012. For large enterprises with complex database environments and database deployment, migration to public cloud has been a secondary or tertiary consideration as our customers recognize it must fit within a much broader set of requirements, planning and initiatives. Our work with Amazon and AWS via our Postgres Plus Cloud Database has shown that cloud migrations are a piece of a total migration strategy (more below).
Migration Strategies & Best Practices
We have counseled thousands of customers on Database Migration and in the process have discovered a few important guideposts to both maximize the return and “time-to-value” of change while minimizing risk. Here are a few considerations:
Build to the big picture
- You will migrate across a spectrum of applications (non-mission critical, mission-critical, home grown, third party COTs) and deployment models (on-premises traditional, on-premises virtualized, private cloud, public/hosted managed service cloud, i.e. AWS)
- Our most successful customers had an upfront view about which workloads were best fits for migration. This allowed them to select and architect (sometimes very simply) an overall migration plan with the maximum chance of success.
- In every case, getting early wins and proof points by migrating on premises applications that were well understood and well managed was the biggest driver for strategic success.
Migrate don’t Port
- Rewriting applications can crush the ROI. That’s why we developed database compatibility technology. Don’t rewrite and also don’t simply use incomplete “translators”. “Migrating” the database schema is a small part of the process. The heavy lifting is getting the app-specific packages, functions, and stored procedures to run without change.
- Rewrites take a long time and open up multiple cans of worms – migrations are much faster.
Leverage, don’t scrap
- Migration is not an “all-or-nothing” endeavor. You have valuable investments in infrastructure, process and staff. All should be leveraged as part of an effective transformation.
- You have spent years capturing the business processes that differentiate your company and deliver the efficiencies that make your business profitable. Don’t throw that away; build on it while deriving savings from a move to open source databases.
Engineer the contract migration
- This is the most overlooked part of a systematic approach for database migration. Your proprietary vendors, and in particular Oracle, have engineered their agreements and your contractual relationship with them to maximize your spend and increase the difficulty for you to change.
- Migrating selected sets of workloads to a lower cost alternative such as EDB Postgres, outside of those contracts offers you savings, and leverage in the negotiation for those workloads that demand specific functionality from the proprietary vendors.
- This is not just a technical migration, engage your procurement and financial groups, including the Chief Financial Officer and Chief Procurement Officer and make it part of your company’s overall strategy. This effort will pay off by giving you right sized cost for workloads, but also in gaining you independence and leverage in the management of your other vendor relations.
Take Charge of the Change
The database migration market is now well established and growing quickly. Thousands of our customers have proven that there is an increasingly well-defined model and approach to this change. Lean in and take charge of your database transformation – the returns are well within your control.