macOS release machine version for Node 12+

Ref: nodejs/node#24521 (comment)

We should consider bumping to at least 10.12 ("Sierra") for Node 12 I think. Maybe even just jump to 10.14 ("Mojave"). Xcode does a really good job of maintaining compatibility, and we have the following in common.gypi:

'MACOSX_DEPLOYMENT_TARGET': '10.7',       # -mmacosx-version-min=10.7

I don't believe we've ever had problems reported for using Node on older versions of macOS up to 10.7, it's now newer versions that are the problem as per the linked issue above.

@gdams

@gdams had adding a 10.13 so that we could start using it in our testing. I think we've had some issues on 10.14, but could make sense to add that instead, use it to flush out the issue and then consider using it for builds.

@gdams do you think you'll have time to get to that in the near future?

Does anyone know if bumping 'MACOSX_DEPLOYMENT_TARGET' has any beneficial effects (maybe performance or size improvements?)

Refs: http://gs.statcounter.com/macos-version-market-share/desktop/worldwide

Legend:
OS X 10 beta: Kodiak - 13 September 2000
OS X 10.0: Cheetah - 24 March 2001
OS X 10.1: Puma - 25 September 2001
OS X 10.2: Jaguar - 24 August 2002
OS X 10.3 Panther (Pinot) - 24 October 2003
OS X 10.4 Tiger (Merlot) - 29 April 2005
[OS X 10.4.4 Tiger (Chardonnay)]
OS X 10.5 Leopard (Chablis) - 26 October 2007
OS X 10.6 Snow Leopard - 28 August 2009
OS X 10.7 Lion (Barolo) - 20 July 2011
OS X 10.8 Mountain Lion (Zinfandel) - 25 July 2012
OS X 10.9 Mavericks (Cabernet) - 22 October 2013
OS X 10.10: Yosemite (Syrah) - 16 October 2014
OS X 10.11: El Capitan (Gala) - 30 September 2015
macOS 10.12: Sierra (Fuji) - 20 September 2016
macOS 10.13: High Sierra (Lobo) - 25 September 2017
macOS 10.14: Mojave (Liberty) - 24 September 2018

@ljharb

Maybe a better view:

Ping @ljharb could you share macOS version stats for downloads of node11?

I have no such stats and no way to obtain them; all nvm downloads come from nodejs.org.

Trends continue.

@gdams

@gdams would you have time to add a 10.14. I do think building on it for 12.x would make sense.

Unfortunately, 10.14 is not currently supported in the ESXi version that we have a MacStadium or at least it wasn't when I spoke to them a few months ago. Will ask them again though.

If not 10.14 then we should probably move up to 10.13 for 12.x

I'm trying to sort out what node version should build on what OS X version, with what Xcode version, and from there to figure out what machines should have (can have?) Xcode 10 installed on them.

BUILDING.md says 13.x and master uses xcode 10; 12.x and before uses xcode 8
12.x is going to stay at xcode 8. 13.x is going to start at 8, and switch to 10:
OS X min version is undocumented, but MACOSX_DEPLOYMENT_TARGET is set in common.gypi. It is min 10.10 for 12.x and later, 10.7 for 10.x and 8.x
Xcode 10 requires OS X 10.14 (or .13 for earlier patches)
- https://en.wikipedia.org/wiki/Xcode#Xcode_7.0_-_10.x_(since_Free_On-Device_Development)
current OS X versions and their corresponding (single) Xcode version:

% parallel-ssh -i -h hosts/macos 'pkgutil --pkg-info=com.apple.pkg.CLTools_Executables | grep version'                                            
[FAILURE] release-macstadium-macos10.11-x64-1 (probably 8.2) -- 11.x and later released here
[FAILURE] release-macstadium-macos10.10-x64-1 (probably 7.2) -- 8.x and 10.x released here
[SUCCESS] test-macstadium-macos10.10-x64-1 version: 7.2.0.0.1.1447826929
[SUCCESS] test-macstadium-macos10.10-x64-2 version: 7.2.0.0.1.1447826929
[SUCCESS] test-macstadium-macos10.11-x64-1 version: 8.2.0.0.1.1480973914
[SUCCESS] test-macstadium-macos10.11-x64-2 version: 8.2.0.0.1.1480973914
[SUCCESS] test-macstadium-macos10.12-x64-1 version: 9.1.0.0.1.1508540944
[SUCCESS] test-macstadium-macos10.12-x64-2 version: 9.2.0.0.1.1510905681

FTR, the nearform machines are 2018 mac mini's running mojave 10.14.2, see #1695 for ongoing efforts to get them up.

I think the TL;DR of the above is: Not a single version of our current macos release or test machines can run Xcode 10, so dual-installation of Xcode is impossible

Here's your source of truth:

build/jenkins/scripts/VersionSelectorScript.groovy

Lines 77 to 78 in 821de09

    
           [ /^osx1010/,                       anyType,     gte(11) ], 
        
           [ /^osx1011/,                       releaseType, lt(11)  ],

There's no additional logic in iojs+release and we only have those two release machines 10.10 and 10.11. So what's on them is what will compile Node for those versions. I know 10.11 has Xcode 8 on it, I don't have access to remind myself what's on 10.10 but it'll be the same as the 10.10 machines you've listed, so Xcode 7.

BUILDING.md is wrong for master and 13, as I said in nodejs/node#30043. It's right for 12 (iirc) and I think that section on "what our binaries are built on" is new so it might not be there for previous versions.

I had a wild thought over the weekend of resurrecting a VM what I used to use in my garage to build releases. I have the MacMini still and a VMWare Fusion license, I could do it, but it feels like a step backward to depend on my network again for release builds, we're just getting over that with the Pi's for Node 8.

@jasnell

resurrecting a VM

Yeah, too wild :-). We've the vmware fusion licenses from @jasnell , and @AshCripps is still working on using them to get the nearform machines running.

So I have been doing some testing with building node 12 and 10 on a catalina vm with xcode 11.

The minimum deployment target for both versions is set to 10.10 so I built the binaries on catalina and then tested them on a 10.10.1 OSX machine running xcode 6.

The results of the tests can be seen here:

Node 12:

flora:node acripps$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.10.1
BuildVersion:	14B25
flora:node acripps$ python tools/test.py -J N test/parallel
=== release test-dgram-connect-send-empty-array ===
Path: parallel/test-dgram-connect-send-empty-array
Command: out/Release/node /home/acripps/nodetests/12/node12/node/test/parallel/test-dgram-connect-send-empty-array.js
--- TIMEOUT ---
=== release test-dgram-connect-send-empty-packet ===
Path: parallel/test-dgram-connect-send-empty-packet
Command: out/Release/node /home/acripps/nodetests/12/node12/node/test/parallel/test-dgram-connect-send-empty-packet.js
--- TIMEOUT ---
=== release test-dgram-connect-send-empty-buffer ===
Path: parallel/test-dgram-connect-send-empty-buffer
Command: out/Release/node /home/acripps/nodetests/12/node12/node/test/parallel/test-dgram-connect-send-empty-buffer.js
--- TIMEOUT ---
=== release test-dgram-send-empty-array ===
Path: parallel/test-dgram-send-empty-array
Command: out/Release/node /home/acripps/nodetests/12/node12/node/test/parallel/test-dgram-send-empty-array.js
--- TIMEOUT ---
=== release test-fs-chmod ===
Path: parallel/test-fs-chmod
assert.js:92
  throw new AssertionError(obj);
  ^

AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:

511 !== 420

    at /home/acripps/nodetests/12/node12/node/test/parallel/test-fs-chmod.js:145:12
    at /home/acripps/nodetests/12/node12/node/test/common/index.js:371:15
    at fs.js:1067:9
    at FSReqCallback.oncomplete (fs.js:146:23) {
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: 511,
  expected: 420,
  operator: 'strictEqual'
}
Command: out/Release/node /home/acripps/nodetests/12/node12/node/test/parallel/test-fs-chmod.js
=== release test-fs-promises ===
Path: parallel/test-fs-promises
/home/acripps/nodetests/12/node12/node/test/common/index.js:710
const crashOnUnhandledRejection = (err) => { throw err; };
                                             ^

AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:

511 !== 438

    at doTest (/home/acripps/nodetests/12/node12/node/test/parallel/test-fs-promises.js:232:18) {
  generatedMessage: true,
  code: 'ERR_ASSERTION',
  actual: 511,
  expected: 438,
  operator: 'strictEqual'
}
Command: out/Release/node /home/acripps/nodetests/12/node12/node/test/parallel/test-fs-promises.js
=== release test-dgram-send-empty-buffer ===
Path: parallel/test-dgram-send-empty-buffer
Command: out/Release/node /home/acripps/nodetests/12/node12/node/test/parallel/test-dgram-send-empty-buffer.js
--- TIMEOUT ---
=== release test-dgram-send-empty-packet ===
Path: parallel/test-dgram-send-empty-packet
Command: out/Release/node /home/acripps/nodetests/12/node12/node/test/parallel/test-dgram-send-empty-packet.js
--- TIMEOUT ---
[05:38|% 100|+ 2304|-   8]: Done

Node 10:

flora:node acripps$ python tools/test.py -J N test/parallel
=== release test-fs-chmod ===
Path: parallel/test-fs-chmod
assert.js:84
  throw new AssertionError(obj);
  ^

AssertionError [ERR_ASSERTION]: Input A expected to strictly equal input B:
+ expected - actual

- 511
+ 420
    at fs.lchmod.common.mustCall (/home/acripps/nodetests/10/node10/node/test/parallel/test-fs-chmod.js:145:12)
    at /home/acripps/nodetests/10/node10/node/test/common/index.js:379:15
    at fs.close (fs.js:988:9)
    at FSReqWrap.args [as oncomplete] (fs.js:140:20)
Command: out/Release/node /home/acripps/nodetests/10/node10/node/test/parallel/test-fs-chmod.js
=== release test-fs-promises ===
Path: parallel/test-fs-promises
/home/acripps/nodetests/10/node10/node/test/common/index.js:691
const crashOnUnhandledRejection = (err) => { throw err; };
                                             ^

AssertionError [ERR_ASSERTION]: Input A expected to strictly equal input B:
+ expected - actual

- 511
+ 438
    at doTest (/home/acripps/nodetests/10/node10/node/test/parallel/test-fs-promises.js:229:18)
Command: out/Release/node /home/acripps/nodetests/10/node10/node/test/parallel/test-fs-promises.js
[03:27|% 100|+ 2109|-   2]: Done

The fs test failures are to be expected as that machine runs with a home directory that is a network FS, so asides from some timeouts on 12 both versions run pretty cleanly.

So I was wondering if this means we could possibly upgrade the release machines to more modern os's or possibly reduce the amount of mac resources we use in build by reducing the amount of old os's we have to run? It would help to have more modern release machines if apple decide to be even more stringent around notorization.

Thoughts?

Might be good to test addons as well; are there any troubles with addons compiled against current Node installs when running in binaries from newer setups.

Aside from that I don't really know what else to look for. Maybe we just have to bite the bullet and ship a 13.x with a new compile toolchain and see what breaks (who sequals) and be prepared to roll back quickly if something does. A lot more cowboy than we've done in the past but maybe we just put our faith in Xcode here.

Ok managed to test addons - I had forgetten to build them on catalina so it took me a while to build them and copy them across to the test machine.

Results for Node 12 running on OSX 10.10

flora:node acripps$ python tools/test.py -J N test/addons
[00:27|% 100|+  55|-   0]: Done

Same result for Node 10:

flora:node acripps$ python tools/test.py -J N test/addons
[00:26|% 100|+  50|-   0]: Done

We don't actually have any OS X 10.13 or later machines yet... but when we do these tests are very promising. Continually adding macs is going to hit the limits of what our donors are willing to supply.

If we can get away with building all mac releases on a single machine, that will help limit our machine requirements (and requirements for humans to maintain the machines).

We still want to actually test on a variety of OX X versions, or do we? Do we need to test on every supported OS X version per supported branch? If so, we'll need 2 new test machines for every OS X release.

12.x is OS X 10.11+, and expires in the time of OS X 10.17, so that'll mean at least 14 test machines will be needed.

I notice https://docs.travis-ci.com/user/reference/osx/#macos-version

@nodejs/build It looks like they have a good selection of OS X from 10.10 to 10.15, would it be worth starting to use Travis to do our CI for OS X, instead of directly using macstadium? Are there any issues with this? Other than git node land doesn't currently check Travis build results? Fix for that is WIP, nodejs/node-core-utils#369

Travis uses macstadium behind the scenes (I think), but removing macstadium machine setup and maintenance from the build WG's support tasks would be a real win.

@nodejs/build It looks like they have a good selection of OS X from 10.10 to 10.15, would it be worth starting to use Travis to do our CI for OS X, instead of directly using macstadium? Are there any issues with this?

Well for one we'd need to make sure the jobs can complete within Travis' 50 minute runtime limit (the current Travis builds on Linux do not without a ccache cache (nodejs/node#30469 will help populate the cache)).

Also I think Travis only runs up to three jobs per repository at a time so there are potential bottlenecks if we vastly increase the number of jobs submitted per build.

Originally the PR to add Travis (nodejs/node#21059) included macOS but during review it was decided to limit to Linux.

Warming up the cache is an issue, but it seems the builds that do massive ccache invalidation are mostly V8, which are infrequent, and done by people who understand the system, and can kick the builds a couple times (we do that anyhow for ephemeral failures in Jenkins). Most contributions don't touch the deps.

The concurrency limit is a good point, but we can increase to 10 concurrent jobs for only $448 per month (https://travis-ci.com/plans), or possibly negotiate with them for more resources for free.

$448 is pretty cheap compared to what we are asking of the foundation members, which is to donate build wg members for 2+ days a week which is $400 dollars per day if they get half a US software devs salary.

@richardlau

@richardlau I think there is an interesting point made in the message you linked to where the decision was made:

We can always test OS X on our regular CI.

limit to Linux.

We cannot in fact do that. Or at least, have not yet. Efforts have been ongoing for weeks, but we don't have any OS X 10.13+ or recent Xcode in our regular CI, but Travis does.

cc: @addaleax @mcollina

Warming up the cache is an issue, but it seems the builds that do massive ccache invalidation are mostly V8, which are infrequent, and done by people who understand the system, and can kick the builds a couple times (we do that anyhow for ephemeral failures in Jenkins). Most contributions don't touch the deps.

It's not just the PR's that touch V8. When the V8 update lands any PR that was opened before it landed will not have a cache of the updated V8 artifacts (Travis looks for a cache for the PR first) and therefore any subsequent builds (either that were already queued or triggered via updates to the PR) will need to be kicked (once nodejs/node#30469 lands -- at the moment you have to delete the PR's cache from Travis and then kick the build). But maybe that's something we'd just have to get used to?

Could also use the 10 free parallel builds on Azure https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops since they have 10.13 and 10.14

This issue appears to be out of date wrt. current work in process.

We should consider bumping to at least 10.12 ("Sierra") for Node 12 I think. Maybe even just jump to 10.14 ("Mojave").

Decision is to build for all release lines on Catalina, work to do that is ongoing in #1695 as well as #1732

Xcode does a really good job of maintaining compatibility, and we have the following in common.gypi:

Decision about deployment target setting for Node.js 14.x is ongoing in #2168

That addresses the questions asked in the description, so I'm closing.

If I misunderstand, feel free to reopen and clarify what needs addressing.

Navigation Menu

Repository navigation

macOS release machine version for Node 12+ #1593

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	[ /^osx1010/, anyType, gte(11) ],
	[ /^osx1011/, releaseType, lt(11) ],

macOS release machine version for Node 12+ #1593

Description

Activity

mhdawson commented on Nov 23, 2018

refack commented on Nov 23, 2018

refack commented on Nov 23, 2018

ljharb commented on Nov 24, 2018

refack commented on Feb 7, 2019

mhdawson commented on Feb 7, 2019

gdams commented on Feb 8, 2019

mhdawson commented on Feb 8, 2019

sam-github commented on Oct 21, 2019

sam-github commented on Oct 21, 2019

rvagg commented on Oct 21, 2019

sam-github commented on Oct 22, 2019

AshCripps commented on Nov 14, 2019

rvagg commented on Nov 15, 2019

AshCripps commented on Nov 15, 2019

AshCripps commented on Nov 15, 2019

sam-github commented on Nov 15, 2019

richardlau commented on Nov 15, 2019

sam-github commented on Nov 15, 2019

sam-github commented on Nov 15, 2019

richardlau commented on Nov 15, 2019

nschonni commented on Nov 15, 2019

sam-github commented on Feb 19, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions