When you run perl Makefile.PL, a Makefile is created. This Makefile includes the same compilation options that were used to build Perl itself. They are stored in the Config.pm module and can be displayed with the Perl -V command. All these options are reapplied when compiling Perl modules.
If you use a different compiler to build Perl extensions, chances are that the options this compiler uses won't be the same, or they might be interpreted in a completely different way. So the code may not compile, may dump core, or may behave in unexpected ways.
Since Perl, Apache, and third-party modules all work together under mod_perl, it's essential to use the same compiler while building each of the components.
If you compile a non-Perl component separately, you should make sure to use both the same compiler and the same options used to build Perl. You can find much of this information by running perl -V.