How to “include” shell source in Perl script

June 27, 2017

Often times you’d want to share a set of environment variables between shell scripts and Perl scripts. Suppose that our variables are defined in a script called “etc/env”:

FOO=BARoo
GURGLE="throbbozongo"

In shell, we can easily include our “env” script with “source” command:

#!/bin/sh
. etc/env
# Or equivalent but more readable:
source etc/env

Right, but how do we include shell script in Perl script? The obvious solution is to read the file, split on /=/ and assign the variables, easy peasy:

open my $env, '<', 'etc/env';
while (<$env>) {
    chomp;
    my ($var, $value) = split /=/;
    $ENV{$var} = $value if $var;
}
close $env;

Uh, wait a bit… $ENV{GURGLE} is now double quoted? Ha, that’s easy too:

...
   $ENV{$var} = ($value =~ s/^['"]|['"]$//gr) if $var;
...

A couple more iterations to account for matching single/double quotes and we may be happily on our way. However we will soon find out that this approach is too naïve to survive the first contact with the Real World: Bourne shell has many quirks and string quotation/interpolation is among the weirdest. Do we really want to account for cases like this?

DISCLAIMER='I '"positively"' can'\''t and won'\''t '"parse"' that!'
QUX=`echo "$FOO" | tr [:upper:] [:lower:]`
SQL="
select foo from bar where baz = '$QUX';
"

These are relatively innocent examples by a long shot, and over time even more interesting stuff will creep into your “env” scripts. Hey it’s Bourne shell, anything goes!

Right, uh… Right… Let’s use shell to parse shell scripts! Yeah!

#!/usr/bin/perl

use strict;
use warnings;

# set -a is to auto-export all variables
my $env = `set -a; source etc/env; env`;

for my $line (split /\n/, $env) {
    # see above...
}

But something’s wrong. Some variables not set? We’re looking inside $env and see this:

...
SQL=
select foo from bar where baz = 'barbaz';

QUX=barbaz
...

What? What?! Oh right, we’re splitting on /=/… That’s why we have $ENV{SQL} that is an empty string and $ENV{select foo from bar where baz } that contains ” ‘barbaz’;”…

I could go on and on with that but I believe I should have proven my point by now: parsing shell scripts is not an option. Not even a last resort option. It’s just too damn hard!

Ok ok you got me you say. Let’s do it the other way around, by including our Perl code in Bourne shell script instead, using the old arcane incantations we found in the depths of perlrun. Note “-a” argument to /bin/sh; this is to export all defined environment variables automatically:

#!/bin/sh -a
#! -*-perl-*-

. etc/env

eval 'exec perl -x -S $0 ${1+"$@"}'
    if 0;

print STDOUT "THIS IS PERL BABY!\n";

Eh, not really…

$ bin/foo
syntax error at bin/foo line 4, near "."
Execution of bin/foo aborted due to compilation errors.

Grr! This is getting ridiculous! Let’s use our Dark Bourne Shell Powers we’ve learned in the Battle of the Quotes to fool both interpreters:

#!/bin/sh -a
#! -*-perl-*-

# This line is evaluated to '. etc/env' in shell and does what we need,
# i.e. includes the etc/env script; in Perl this line evaluates to
# concatenation of two strings into "etc/env" and effectively does nothing.
"". "etc/env";

eval 'exec perl -x -S $0 ${1+"$@"}'
    if 0;

print STDOUT "THIS IS PERL BABY!\n";

Now behave, you piece of electronic crap!

$ bin/foo
THIS IS PERL BABY!

MUA-HA-HA! Now you see! But wait, now you don’t…

$ perl -d bin/foo

Loading DB routines from perl5db.pl version 1.49_05
Editor support available.

Enter h or 'h h' for help, or 'man perldebug' for more help.

/bin/sh: -d: invalid option
Usage:	/bin/sh [GNU long option] [option] ...
	/bin/sh [GNU long option] [option] script-file ...

(facepalm) (facepalm) (facepalm) That shebang… It’s a shell script, remember? Well our old pal Perl is just trying to be helpful and calls the “right” script interpreter for us… Except that it’s not.

Ok ok. You see where I’m getting at, right? Let’s peruse the arcane volume of perlrun once more, and read right below the place where we found the eval incantation:

       If the "#!" line does not contain the word "perl" nor the word "indir"
       the program named after the "#!" is executed instead of the Perl
       interpreter...

The word “perl” nor the word “indir”… Uh, what if?..

#!/bin/sh -a #indir
#! -*-perl-*-

"". "etc/env";

eval 'exec perl -x -S $0 ${1+"$@"}'
    if 0;

print STDOUT "THIS IS PERL BABY!\n";

WHOA! IT WORKED!

$ bin/foo
THIS IS PERL BABY!

$ perl -d bin/foo

Loading DB routines from perl5db.pl version 1.49_05
Editor support available.

Enter h or 'h h' for help, or 'man perldebug' for more help.

main::(bin/foo:4):	"". "etc/env";
DB<1> R
Warning: some settings and command-line options may be lost!

Loading DB routines from perl5db.pl version 1.49_05
Editor support available.

Enter h or 'h h' for help, or 'man perldebug' for more help.

main::(bin/foo:4):	"". "etc/env";

And we don’t even need -x -S parameters as well as the pretty but unnecessary
“-*-perl-*-” comment/decoration, since we’re effectively executing the whole
script in Perl anyway. So the final incantation looks like this:

#!/bin/sh -a #indir

"". "etc/env";

eval 'exec perl $0 ${1+"$@"}'
    if 0;

print "sql: $ENV{SQL}\n";
print "disclaimer: $ENV{DISCLAIMER}\n";
print "argv: " . (join ' ', @ARGV) . "\n";

Lo and behold!

$ bin/foo bar baz
sql: 
select foo from bar where baz = 'barbaz';

disclaimer: I positively can't and won't parse that!
argv: bar baz

Neat huh? Happy coding! ;)

tags: , ,
posted in Software development by nohuhu

Follow comments via the RSS Feed | Leave a comment | Trackback URL

Leave Your Comment

 
Powered by Wordpress and MySQL. Theme by Shlomi Noach, openark.org