Mac2K: The Years’ 2038 and 2040 Problems

[B S Magnet]

Mac2K: The Years’ 2038 and 2040 Problems
WikiPost struck: 31 January 2023
Last edited: 31 January 2023

---

OVERVIEW

For folks who use PowerPC and even Intel Macs (prior to, say, macOS 10.15 Catalina), it’s never too soon or too late to start planning for the impending, dual Y2K-like hard limits of the POSIX-based “Year 2038” — or Y2K38 — 32-bit problem, or the uniquely Apple-specific Hierarchical File System (HFS/HFS+) “Year 2040” — or Y2K40 — problem.

This thread, which for the moment is going to be a WikiPost placeholder (for more, undoubtedly, to be appended later by MR forum folks), invites that discussion to help us to manage the continued use of our vintage Macs beyond both 2038 and 2040.

Our PowerPC and Early Intel Macs may be “old” by whatever metric one uses, but consider how that’s only fifteen years from now. We know folks who are stll running original, 68K-based Macintoshes right at this moment (including, possibly, you reading this!). The eldest surviving, running examples of the Macintosh 128K are now nearing forty.

---

MACS (PARTLY OR COMPLETELY) IN THE CLEAR

We already know Mac OS 9.2.2 and some earlier, Classic versions of Mac OS are unaffected by the 32-bit-based POSIX bug.

Mac OS, prior to Apple’s integration of the NeXTSTEP OS in 1997, was not based on the UNIX epoch (which began 1 January 1970 at 00:00:00 UTC; the earliest date POSIX32 can register is 13 December 1901 at 20:45:52 UTC, whereas the latest is 19 January 2038 at 03:14:07 UTC — basically, pi-o’clock! 🥧, but sadly no 🍏🥧 for our systems which rely, in some part, on 32-bit addressing).

Unfortunately, HFS and HFS+ are not a part of that, which means both Mac OS 9 and Mac OS X have another barrier. File and directory dates on HFS/HFS+ are limited to the same, 32-bit-based problem shared with POSIX — albeit with a epochal twist.

You’ve probably seen this at least once whilst stumbling upon a file creation/modification date of 1 January 1904. The other end of this HFS epoch is set to arrive on 6 February 2040 at 06:28:15 UTC — a Monday morning, no less.

For folks running some iteration of macOS 10.13 and later on their Intel Macs (going as far back as selected Penryn Core 2 Duo models from 2008, or even earlier Macs with swapped-out CPUs), much of the Y2K40 HFS+ problem is obviated by running APFS, in lieu of HFS+, for your boot volume.

Folks on macOS 10.15 Catalina and later, meanwhile, don’t need to worry about running across 32-bit code (as Apple phased out system ability to run 32-bit binaries after macOS 10.14 Mojave). Moreover, macOS 10.15 and later isn’t designed to recognize HFS+ volumes, but 10.15 and later should be able to read other, 64-bit-based file systems.

---

THIS THREAD’S GOALS

This WikiPost thread is being struck to facilitate discussion, collaboration, and fixes for both the Y2K38 and Y2K40 problems as they affect our Macs.

Simple solutions for selected situations notwithstanding (see examples from the previous section), there will be circumstances for which simple solutions cannot remedy. Fixes could come in the form of patches to software or even to firmware, or in migrating HFS/HFS+ volumes to non-APFS-related formats which older Macs can continue to read and/or write.

---

MORE TO BE ADDED HERE

This, for the moment, is a placeholder post for bookmarking. It’s up to you all to bring together what you know.

 

[f54da]

See also thread this was forked from, https://forums.macrumors.com/thread...re.2378689/page-2?post=31927110#post-31927110 where I suggest how we can patch HFS to bsd timestamp conversion to fix this.

 

[theMarble]

Crazy to think that by the time the HFS+ epoch is hit, Mac OS X would be just under a month away from it's 40th birthday!

For folks running some iteration of macOS 10.13 and later on their Intel Macs (going as far back as selected Penryn Core 2 Duo models from 2008, or even earlier Macs with swapped-out CPUs), much of the Y2K40 HFS+ problem is obviated by running APFS, in lieu of HFS+, for your boot volume.

Can Sierra also use APFS boot volumes? Or was it just non-bootable volumes back then?

 

[B S Magnet]

Crazy to think that by the time the HFS+ epoch is hit, Mac OS X would be just under a month away from it's 40th birthday!

Even older. Mac OS X Server, or Rhapsody, went on sale 16 March 1999. The familiar, Aqua-derived UI, meanwhile, is slightly younger.

Can Sierra also use APFS boot volumes? Or was it just non-bootable volumes back then?

By default, APFS was introduced in Sierra, but unless I’m mistaken, the extent of support released publicly was for read-only access.

 

[Wowfunhappy]

Noting for the new thread that my Mavericks Virtual Machine seems to work fine with the year set to 2040. I didn't test it that extensively but it boots, applications open, I can save files, etc.

 

[f54da]

^^ Any files saved will have the last modified/creation timestamp as unix epoch, but other than that things should continue work. So probably not an issue for casual one-off use, but if you're planning to use it in a meaningful capacity you'll effectively lose all date resolution. It should be a trivial fix, but I'm too lazy to do it now, ping me a decade from now and I'll whip up something then if nobody gets to it first.

Also note that it's not just creation/modified dates, linux/osx also store a last access timestamp, but really you should be using noatime anyhow (ha!)

 

[Doq]

It should probably also be obvious but if you're not running Mac OS X or an HFS(+) configuration then you're probably also not going to run into Mac2K.

Linux really is the future.

 

[B S Magnet]

It should probably also be obvious but if you're not running Mac OS X or an HFS(+) configuration then you're probably also not going to run into Mac2K.

Which sounds like a good time to give MorphOS a spin!

Linux really is the future.

Are there Linux patches for Y2K38 on 32-bit linux boxes?

 

[Doq]

Are there Linux patches for Y2K38 on 32-bit linux boxes?

Kernel 5.6 and later uses a 64-bit time_t, even on 32-bit kernels. It does require all of userspace to also be in 64-bit time_t, but most reasonable modern deployments should be okay.

LKML: Arnd Bergmann: [GIT PULL] y2038: core, driver and file system changes

lkml.org

 

[Slix]

This is interesting to learn, given that I just saw this ad recently. I guess it was not entirely true that they worked "perfectly" with years up to 29940.

 

[B S Magnet]

This is interesting to learn, given that I just saw this ad recently. I guess it was not entirely true that they worked "perfectly" with years up to 29940.
View attachment 2151485

WOMP WOMP… 🎺

As a former (recovering) copywriter, you get tasked with finding whatever way you can to thread a compelling-sounding fact or idea through clownishly narrow needle eyes. That’s the craft. That’s what gets the client to sign your cheque.

 

[Mr. Dee]

I’m adding a calendar note, will probably do a temporary work around by 2037 such as resetting the clocks by 15 years.

 

[Doq]

Here is a non-exhaustive list of Linux distributions and their state of being Y2K38-safe. Generally speaking, distros that shipped with Linux Kernel 5.6 or newer should be okay, with patches existing for 5.4. I only looked into popular options; exercise caution with more obscure picks. Shipping kernel may not be the latest kernel available for that distribution, particularly those with a more liberal kernel update cycle.

Spoiler: Table of Y2K38-safe Linux Distros

Distribution	Version/Date	Shipping Kernel
Debian	11.0 (bullseye) (14 Aug 2021)	5.10.140
Debian	unstable (sid)	6.0.12
Ubuntu	16.04 (21 Apr 2016)	2.6.24 updateable to 5.4.0 unofficially
Void	Current (25 Aug 2021)	5.4.227 updateable to 6.0.13
ArchPOWER	Current (10 Dec 2022)	5.10.158
Gentoo	always and forever	5.15.88

 

[Amethyst1]

Moreover, macOS 10.15 and later isn’t designed to recognize HFS+ volumes, but 10.15 and later should be able to read other, 64-bit-based file systems.

HFS+ is still usable in 10.15 and later versions.

I guess it was not entirely true that they worked "perfectly" with years up to 29940.

How did they come up with that number?

 

[Smartuser]

This really keeps me up all night, every night.

 

[iAppleOrchard]

Oh no... the world is going to end in 2038!!!

In all seriousness, I think Apple should do everything they can, even if they have to update 20 year old software.

 

[robertosh]

I will focus on reaching those years. Alive, I mean

 

[mectojic]

I don't think this thread doesn't quite detail the actual consequences of the Y2K38 problem. Can we clarify, as I'm actually not sure how major this problem is.

Eg.1) You save a file on your HFS+ Mac. The modified date will show as 1904. Annoying, but not world-ending.

Eg.2) Someone sends you a file in 2045 which you open on your HFS+ Mac; the file presumably will say it's made in 190x? Annoying, but still fine. Perhaps someone could create a little reference script that helps us track dates, and convert back and forth as needed.

Eg.3) Web browsing requires system clock to be up to date. Ok, we need a fix for that, assuming we are still bothering with heavily patched semi-modern web distros by then, and the entire internet isn't broken.

Eg.4) You choose "set date and time based on Apple server". What happens then? Does it revert to 1904+ and continue to sync?

Eg.5) ? Any other time-based systems? Certain application registrations tend to be fine, if they're vintage then the date can just be set back to 1990s/2000s again.

 

[f54da]

1) Will show as 1970. time_t cannot represent anything before unix epoch,
2) Same as above. In fact it's slightly worse since any file modified will have its last modified time overwritten to unix epoch as well. So if you care about last modified times being meaningful, you will be destroying metadata. This is concerning enough to me that I will try to have an updated hfs kext ready a year or two before, to give every time to test things out and make sure they are ready (if no one else does it first). I suspect since OSX in the future will still support reading HFS, apple will probably do the same patch where they reallocate (1904, 1970) range.



3) XNU clock works perfectly fine, so any 64-bit apps will be ok. Web browning probably OK as well, but at the rate internet standards change 10 years from now I suspect you may as well be using lynx or gopher anyway.
4) Unknown how mavericks will work with ntp, but considering that setting date directly via terminal works, worst case you can run your own daemon or something to periodically update things.
5) Any 32-bit carbon apps will likely to be impacted.

 

[swamprock]

On a mostly on-topic note, I should dig out my old TRS-80 Model III and see what happens if I enter a 2038 date 😂

(If it boots, that is...)

 

[f54da]

These patches to `to_bsd_time` and `to_hfs_time`in the `bsd/hfs/hfs.h` kext should do it. Anyone want to double check my work?

Also I was mistaken seems like HFS support is built directly into xnu, the kext is only for fsck. Luckily fsck only converts from unix time _to_ hfs time (just the addition of MAC_GMT_FACTOR) so we just let it wrap. There is one case in fsck that goes the other way, when it mounts a volume it gets the volume creation date, but I think it's OK to just let this map onto the unix epoch, how often do you check when your volume was created anyway...

Anyone brave enough want to patch `bsd/hfs/MacOSStubs.c` in the kernel source and try this out (in a VM unless you don't mind your data possibly getting corrupted).

I don't know offhand the instructions to build XNU from source. @Wowfunhappy I recall you did this once, if you still remember, could you post the instructions here as well as the xnu branch you used?

#include <stdio.h>
#include <assert.h>

/*
*    This is the straight GMT conversion constant:
*    00:00:00 January 1, 1970 - 00:00:00 January 1, 1904
*    (3600 * 24 * ((365 * (1970 - 1904)) + (((1970 - 1904) / 4) + 1)))
*/
#define MAC_GMT_FACTOR        2082844800UL
#define YEAR_2038_CUTOFF

typedef long time_t;

/*
* to_bsd_time - convert from Mac OS time (seconds since 1/1/1904)
*         to BSD time (seconds since 1/1/1970)
*/
time_t to_bsd_time(u_int32_t hfs_time)
{
    // Preserve legacy behavior that uninitialized timestamps (0) return back 0 again
    // NOTE: This means that technically we will never be able to represent a file whose timestamp
    // is exactly at the HFS overflow second, since it will be treated the same as an uninitialized timestamp.
    // I recommend you don't create any files at that very second, maybe even shut the laptop down for an hour so that access times
    // don't get reset.
    if (hfs_time == 0) return 0;
  
    time_t adjusted_hfs_time = hfs_time;
    // An hfs timestamp that is detected to be before 1970 will be mapped onto 2038+.
    if (adjusted_hfs_time < MAC_GMT_FACTOR) {
        adjusted_hfs_time += ((long) 0xffffffff) + 1; // add uint32_max + 1 to any timestamp that is detected to have wrapped around
    }
  
    return adjusted_hfs_time - MAC_GMT_FACTOR;
}

/*
* to_hfs_time - convert from BSD time (seconds since 1/1/1970)
*         to Mac OS time (seconds since 1/1/1904)
*
* To solve the year 2038 issue, bsd timestamps after 2038
* will be mapped onto [1904, 1970) when stored in HFS time
*/
u_int32_t to_hfs_time(time_t bsd_time)
{
    u_int32_t hfs_time = (u_int32_t)bsd_time;
  
    /* don't adjust zero - treat as uninitialzed */
    if (hfs_time != 0) {
        hfs_time += MAC_GMT_FACTOR;
        // Note that this may overflow a uint32 and wrap around. That's OK. We will treat
        // an HFS time between (0, GMT_Factor) and representing 2038+
        // Obviously this won't work for 2104+ onward, but by then anyone reading this will not have to worry about it...
    } 
    return (hfs_time);
}


int main(int argc, char *argv[]) {
  
    // https://www.epochconverter.com/ is handy
    // and https://www.epochconverter.com/mac
  
  
    long unix_time = 1675316522L; // Feb 2, 2023
    printf("Unix time: %ld\n", unix_time); // Of course HFS can handle this fine
    printf("HFS Time: %u\n\n", to_hfs_time(unix_time));
    assert(to_bsd_time(to_hfs_time(unix_time)) == unix_time);
    assert(to_hfs_time(to_bsd_time(to_hfs_time(unix_time))) == to_hfs_time(unix_time));

  
    unix_time = 2177471200L; // Jan 1, 2039 (in the evening or so....)
    printf("Unix time: %ld\n", unix_time);
    printf("HFS Time: %u\n\n", to_hfs_time(unix_time)); // This is OK as far as HFS is concerned, hfs time can represent this
    assert(to_bsd_time(to_hfs_time(unix_time)) == unix_time);
    assert(to_hfs_time(to_bsd_time(to_hfs_time(unix_time))) == to_hfs_time(unix_time));
  
    unix_time = 2212290400L; // Feb 8, 2040. HFS cannot represent this natively since it would overflow an uint32_t
    printf("Unix time: %ld\n", unix_time);
    printf("HFS Time: %u\n\n", to_hfs_time(unix_time)); // Should map to Jan 1904
    assert(to_bsd_time(to_hfs_time(unix_time)) == unix_time);
    assert(to_hfs_time(to_bsd_time(to_hfs_time(unix_time))) == to_hfs_time(unix_time));
  
    unix_time = 2527391200L; // Feb 2, 2050
    printf("Unix time: %ld\n", unix_time);
    printf("HFS Time: %u\n\n", to_hfs_time(unix_time)); // Should map to Dec 1939
    assert(to_bsd_time(to_hfs_time(unix_time)) == unix_time);
    assert(to_hfs_time(to_bsd_time(to_hfs_time(unix_time))) == to_hfs_time(unix_time));

    // Preserve behavior that uninitialized timestamps map onto each other
    unix_time = ((long) 0xffffffff) + 1; // HFS overflow
    printf("Unix time: %ld\n", unix_time);
    printf("HFS Time: %u\n\n", to_hfs_time(unix_time)); // This overflows to zero, so we cannot hope to distinguish it from uninitialized timestamp.
    assert(to_hfs_time(unix_time) == 0);
  
  

    // Preserve behavior that uninitialized timestamps map onto each other
    unix_time = 0L; // Epoch
    printf("Unix time: %ld\n", unix_time);
    printf("HFS Time: %u\n\n", to_hfs_time(unix_time));
    assert(to_bsd_time(to_hfs_time(unix_time)) == unix_time);
    assert(to_hfs_time(to_bsd_time(to_hfs_time(unix_time))) == to_hfs_time(unix_time));
}

Edit: Also for completeness if you use a kernel built from source you will lose the ability to do iMessage. I suspect this likely won't matter by 2040 anyway, but it's a heads up. If you really wanted you could theoretically hotpatch the kernel but I highly suspect to_bsd_time might have ended up inlined so it'd be messy. If it's not inlined it's easier, but hotpatch is still messy.

 

[Wowfunhappy]

XNU Build Instructions for 10.9.5 are at: https://gist.github.com/Wowfunhappy/2fca7ed5d1b1310de3854b92c65eef7e

(Sorry if they're a bit terse, I wrote them for my own reference, let me know if you have questions.)

Overall I found compiling XNU to be shockingly quick and easy. The trickiest part was that you do seem to need the exact version of xcode referenced in the gist.

Also if you recompile XNU make sure to also fix Apple's dumb KQueueScanContinue bug while you're in there! https://github.com/blueboxd/chromium-legacy/issues/44

 

[f54da]

Based on comment in http://shantonu.blogspot.com/2013/10/building-xnu-for-os-x-109-mavericks.html building with 5.1.1 or later will work if you "occurrences of -Werror from makedefs/MakeInc.def" which makes sense I guess clang might have changed defaults.

Also was it ever figured out why the size of kernel built from source differs from prebuilt (besides the private IOPower functions). I don't care much about iMessage but it'd bother me if something else crucial like power management was not included. Cursory digging shows XCPM is not included on newer xnu versions, but I don't know about mavericks xnu. I need to dig more...

 

[f54da]

Oh this is not good... starting from 10.8.5 onwards XCPM is indeed missing. OSX uses XCPM for ivy bridge and newer, so if you have a 2014 MBP+ and use xnu from source, your processor will be stuck running at its base frequency instead of speedstepping as it should.

Why doesn't Apple release full kernel source code?

Apple releases their XNU kernel source code for macOS with each minor release of the Operating System covered under the APSL Version 2 at their open source website. The license terms can be found h...

opensource.stackexchange.com

Xnu CPU Power Management

Some time ago I blogged about XCPM aka Xnu CPU Power Management and I have been digging ever since the first Developer Preview of Mavericks, but I never shared what I know about it. And now that OS…

pikeralpha.wordpress.com

Need to see what happens if you disable xcpm in boot args, will it fall back to appleintelpowermanagement kext or not.

@Wowfunhappy you use hackintosh right, how do they deal with this there? What architecture processor are you using (e.g. haswell)?

 

[f54da]

Ah but good news. `to_bsd_time` is _not_ inlined.

                     _to_bsd_time:
ffffff8000591500         push       rbp
ffffff8000591501         mov        rbp, rsp
ffffff8000591504         cmp        edi, 0x7c25b081
ffffff800059150a         jb         0xffffff8000591514

ffffff800059150c         add        edi, 0x83da4f80
ffffff8000591512         jmp        0xffffff8000591516

ffffff8000591514         xor        edi, edi                                    ; XREF=_to_bsd_time+10

ffffff8000591516         mov        rax, rdi                                    ; XREF=_to_bsd_time+18
ffffff8000591519         pop        rbp
ffffff800059151a         ret

And from what i can see, we even have 5 extra bytes at the end of the procedure that are unused. So I think we can either hand patch it or insert a trampoline.

Total we have 32 bytes to work with. Someone want to codegolf the previous C code to fit into 32 bytes? Worst case we could trampoline it, but I'd rather not do that, it feels messy. The best clang and gcc can do is ~36, I'm sure someone here can squeeze out some more with some bit magic.

If we remove the ` if (hfs_time == 0) return 0;` check then we're down to 28. I don't know how critical this check is though.

Edit: slight reshuffling of order of ops gets us down to 28

    if (hfs_time == 0) return 0;
 
    time_t adjusted_hfs_time = hfs_time - MAC_GMT_FACTOR;
    // An hfs timestamp that is detected to be before 1970 will be mapped onto 2038+.
    if (adjusted_hfs_time < 0) {
        adjusted_hfs_time += ((long) 0xffffffff) + 1; // add uint32_max + 1 to any timestamp that is detected to have wrapped around
    }
 
    return adjusted_hfs_time;

to_bsd_time:
        xor     eax, eax
        test    edi, edi
        je      .L1
        mov     edi, edi
        mov     rax, rdi
        sub     rax, 2082844800
        jns     .L1
        mov     eax, 2212122496
        add     rax, rdi
.L1:
        ret

Edit 2: Hold on doesn't this work just as well? We're basically just working in the equivalence class modulo 1+UINT32_MAX.

#define INT_MAX_WRAP (((long) 0xffffffff) + 1)


 if (hfs_time == 0) return 0;
  
    time_t adjusted_hfs_time = hfs_time;
    // An hfs timestamp that is detected to be before 1970 will be mapped onto 2038+.
    return ((adjusted_hfs_time - MAC_GMT_FACTOR) + INT_MAX_WRAP) % INT_MAX_WRAP;

12 bytes with clang -Os. I think as the compiler realized, doing the modulus in time_t is unnecessary and you could just do "(time_t)(hfs_time - MAC_GMT_FACTOR)` relying on the fact that allowing overflow uint32_t already gives you the equivalence class modulo 2^32. But I like being explicit I guess.

        lea     eax, [rdi - 2082844800]
        test    edi, edi
        cmove   eax, edi
        ret