No Comments

The 2038 problem and how to solve it for MongoDB

Rajiv Thapar | APAC Partner Director, MongoDB
India, 16 Apr 2019
MongoDB-New

by , , No Comments

16-Apr-2019

This year, 2019, is halfway between 2000 and 2038. If you don’t know, 2038 is going to be an interesting year like 2000 was an interesting year for dates and times. 2038 is the year that the 32-bit signed integers that people have been using since the 1970s to represent time will roll over; 2,147,483,647 seconds will have passed since 1 January 1970 and rolling over means the signed value flip to the largest negative value.

As negative values are used to represent the years before 1970, some systems will suddenly time travel into 1901. Others will trip up with date calculations or logic and make new errors. It all makes for a lot of software breaking in 2038. But you don’t have to wait till 2038 to enjoy these problems. If your applications deal with dates in the future, especially up to 20 years in the future, and you use a 32-bit time field then some of your calculations may already be overflowing.

Do I Have A Problem?

First the good news. You should already be using 64-bit dates in MongoDB unless a design decision was taken to use old Unix style 32-bit dates. Why would someone have done that though? Let’s have a look:

  • A time series database includes a lot of timestamps so the opportunity to save 4 bytes per timestamp is quite tempting when you are dealing with hundreds of millions of timestamps.
  • An application interacting with a legacy app may store 32-bit dates given to it by the legacy app. The legacy app will likely need updating, but that’s no reason to leave the database with data that could cause problems in 2038.

About Date

The JavaScript Date object is what MongoDB uses its basis for date handling. It’s usually wrapped as an ISODate() for portability. The Date() object is a 64-bit date representation which works in milliseconds, unlike 32-bit Unix time which works in seconds. Those extra 32 bits allow Date to store the millisecond precision and -100,000,000 days to 100,000,000 days relative to Jan 1st, 1970. That’s 273,972 years roughly, long enough for pretty much anything unless we invent FTL space travel. By being based on the same time for 0, it’s also easy to convert between Unix 32-bit times and dates. If I just hop over to my command line…

date1

That’s the date “now” in 32-bit Unix time. Now if I start up node (or open the JavaScript console in my browser…)

date2

That is not “now”. The Date class can take a value in a constructor, but it has to be a time in milliseconds. So, let’s multiply that by 1000 before handing it to Date.

date3

Now we have it, if we want to get the 32 bit Unix date time, we can get the value and divide by 1000.

date4

Word of warning though, don’t make assumptions this will work. After 2038, that calculation will return an invalid value. Make sure you do some validation before you try and covert back.

About ISODate

Now in MongoDB, as I said, we use Date as the foundation for date handling, but we wrap it in ISODate for portability. Let’s just wake up the mongo shell and try the above with MongoDB:

date5

That error is because ISODate’s are meant to be created with formatted strings of date and time, not with integer or long values. But it’s not a problem because in the shell, if you create a Date…

date6

… you get an ISODate back anyway. ISODate has all the Date methods too, so valueOf() works.

As a bonus, when you save an ISODate, it is typed so browsing the data will show you human-readable:

date7

That’s so much easier to parse than an apparently arbitrary integer which is how 32-bit dates are typically stored.

Actions for Tomorrow

Now you know this, you may want to consider regularly reviewing your own code to make sure that you aren’t importing 32-bit dates and keeping them as 32-bit integer dates. It’s simple to convert them to 64-bit values as we’ve shown. The 4 extra bytes per date versus a massive migration headache should be an easy decision to make. Also, look out for those 32-bit integer dates leaving your application and heading to other components; you can convert the dates back down to 32-bits but make a note where those down conversions are happening.

Underlying this all is a simple message, future proof now and make all dates 64-bit ISODates(). You’ll not only win peace of mind but easier to read dates in your database.

0
0

  • Rajiv Thapar has over two decades of experience in IT Business Value Chain with exposure to General Management with P&L responsibility, Direct and Channel sales in Commercial and Consumer space, Product Management and Project Management.

More From Rajiv Thapar | APAC Partner Director, MongoDB :
16-Apr-2019