json - "Fatal error: 'EXTERN.h' file not found" while installing Perl modules

Question

While trying to install Perl modules like JSON::XS or YAML::XS, i receive the same error:

XS.xs:1:10: fatal error: 'EXTERN.h' file not found

I use MacBook, xCode is up to date, everything else that could help is up to date too.

Answer 1

Since OS X El Capitan, Apple introduced System Integrity Protection which restricts writing to /usr/lib /usr/bin and other sensitive directories (even to root or sudo user) that are used by the installation of Perl bundled with the Operating System. This can cause issues when it comes to installing new modules and also if trying to install XS modules ( those linked to external C libraries ).

For this reason you should not consider the default Perl installation as a working development environment, especially if you are installing custom modules.

Check out this thread on PM and others. I had since El-Capitan managed to solve this before by manually building from tarball and adding a few params or environment variables to set the paths believing that it would be best to retain use of the system Perl but this is not the way to go. This makes your environment difficult to build but also brittle and sensitive to OS updates that may either break things in many different ways.

The best practice seems to be starting with a Perl using brew install perl and work in this environment, remembering to setup your bash_profile as directed by the installer.

Also worth remembering to do a brew link perl. If you receive warnings about this clobbering what looks like system Perl libraries don't worry - these are likely modules that were installed by you over the top and it will cause you less trouble to link over these. If you have concerns, make a note of which module installs will be cleared and re-install them once your environment is configured ( ie your module installer approach is configured using cpanm or sticking with the old perl -MCPAN -e shell etc)

This new Perl setup from brew eliminates the need to continuing running sudo which adds another layer of things that can go wrong as environment variables don't follow through and permission conflicts arise etc.

Finally to simplify package/module installation I suggest doing a brew install cpanminus. If you had previously already installed this, you can ensure the paths etc are configured by doing a brew reinstall cpanminus

If you want to take it another step further then you can install perlbrew as well which will give you the ability to run multiple versions of Perl as your user and configure these with their own libs and modules which can be very useful particularly if aligning with your production environment for testing etc.

One problem you may face if moving from system Perl to this kind of approach is needing to deal with any hangovers from installing things with sudo. It wis worth taking a little time to get all this set up right though and your issues going forward will be greatly reduced and you won't be left with that nagging feeling that you don't want to change anything for fear of it all breaking.

I have also come across a Perl Blog Article that suggests a fix for XS issues with perlbrew on Mojave

This Gist described updating your cpan shell install root though this shouldn't be necessary unless your cpan is stuck in an old config after taking steps above.

I've also raised this as a new issue on PerlMonks