Connect to a PostgreSQL database using Perl DBI

Objective

To establish a connection to a PostgreSQL database using the Perl DBI module

Background

The DBI module is an abstraction mechanism by which a Perl script can interact with a database with minimal regard for which DBMS (in this case PostgreSQL) is being used to host the database.

In order to use a particular database it is first necessary to connect to it. This gives you a connection handle which is needed when calling other DBI functions.

Scenario

Suppose that there is a database called finance hosted using PostgreSQL on a machine called db.example.com. It accepts remote TCP connections on the usual port number (5432). Valid credentials are the username ‘user’ and the password ‘xyzzy’.

Prerequisites

These instructions assume that you have:

• a working installation of PostgreSQL, and
• a working installation of Perl.

For this particular scenario, PostgreSQL must be configured to accept remote TCP connections to the database and username stated above. (The usual default is to allow connections from the local machine only.)

Method

The DBI module itself does not have the ability to communicate with any specific DBMS: for that it is necessary to install the appropriate back-end module, which in the case of PostgreSQL is DBD::Pg.

On Debian-based systems (including Ubuntu) the package that provides this module is libdbd-pg-perl:

apt-get install libdbd-pg-perl

and on Redhat-based systems it is perl-DBD-Pg:

yum install perl-DBD-Pg

In both of these cases, installing the back-end DBD package should automatically install the front-end DBI package as a dependency. Within Perl the dependency relationship is reversed: it is the DBI module that must be loaded explicitly, it then loads any required DBD modules as and when they are needed:

use DBI;

The connection to the database is opened using the function connect. It returns a connection handle, which is needed when making subsequent calls to the DBI module:

my $dbh = DBI->connect('dbi:Pg:dbname=finance;host=db.example.com','user','xyzzy',{AutoCommit=>1,RaiseError=>1,PrintError=>0});

The first argument to connect is a string which specifies the required ‘data source’. For PostgreSQL it should begin with the prefix dbi:Pg:. Following this prefix is a list of settings separated by semicolons. The example above has two of these:

• dbname, the name of the database. It defaults to the username, but you will almost always want to override this.
• host, the name of the database server. This is required when connecting to a remote server; when omitted, a local connection is made using a UNIX-domain socket.

The second and third arguments are the username and password. If the username is undefined then it defaults to the owner of the current process. The password should be left undefined if no password is required.

The fourth argument is a hash ref containing a set of options. The documentation recommends that AutoCommit should always be specified explicitly. For most purposes it should be enabled in the first instance: if particular transactions require it to be disabled then this can be done later.

By default the option RaiseError is false and PrintError is true, meaning that errors are printed as warnings but do not raise exceptions. It is safer to reverse these defaults (as was done above), because then it is not necessary to explicitly check for errors after every DBI function call.

Testing

The script below opens a connection then tests it using a statement that does not depend on the content of the database:

#!/usr/bin/perl

use DBI;

my $dbh = DBI->connect('dbi:Pg:dbname=finance;host=db.example.com','user','xyzzy',{AutoCommit=>1,RaiseError=>1,PrintError=>0});
print "2+2=",$dbh->selectrow_array("SELECT 2+2"),"\n";

It should produce the output:

2+2=4

(Note that a SELECT statement without a FROM clause is not standard SQL, so cannot necessarily be used for testing other types of database.)

Troubleshooting

Provided that you have enabled the RaiseError option (as recommended above), any serious problems should result in either a compilation error or an exception. By default these will be reported to stderr. Some of the more common errors are addressed in detail below.

If you are writing a CGI script then you should look in the web server error log. Alternatively, it may be feasible to execute the CGI script from the command line for testing purposes.

Errors

Can't locate DBI.pm

An error similar to:

Can't locate DBI.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at ./example.pl line 3.
BEGIN failed--compilation aborted at ./example.pl line 3.

could indicate that:

• the DBI module (and therefore probably the DBD module) has not been installed, or
• the module is not on the include path (@INC).

On Debian-based systems the DBI module is provided by the libdbi-perl package, and on Redhat by perl-DBI. In both of these cases it should be installed automatically as a dependency of the DBD package, so if this has not happened then it is quite likely that the DBD module has not been installed either. You can obtain a list of the installed DBI and DBD modules with the command:

dpkg --get-selections | grep "^libdb[id]-"

on Debian and

yum list installed | grep "^perl-DB[ID]"

on RedHat.

An issue with the include path would be unusual if you are using pre-packaged modules and have not overridden the default. You can check by inspecting the content of @INC:

print join "\n",@INC;

Bear in mind that the path seen by (for example) a CGI script will not necessarily be the same as for a script executed from the shell. For this reason, you should try to ensure that any script used for testing is invoked in exactly the same manner as the script being investigated.

Can't locate DBD/Pg.pm

An error similar to:

install_driver(Pg) failed: Can't locate DBD/Pg.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .) at (eval 3) line 3.
Perhaps the DBD::Pg perl module hasn't been fully installed,
or perhaps the capitalisation of 'Pg' isn't right.
Available drivers: DBM, ExampleP, File, Gofer, Proxy, Sponge, mysql.
 at ./example.pl line 5

indicates that the DBI module has been successfully loaded, and that some DBD modules are available (listed on the penultimate line), but that the specific DBD module needed to access PostgreSQL cannot be found. Possible explanations are that:

• the required DBD module has not been installed (see above),
• the module is not on the include path (see above, but unlikely if the DBI module was loaded), or
• the name of the data source passed to DBI::connect was incorrect.

You can check the last point by inspecting the first line of the error message, where it gives the path to the module that could not be loaded. For connecting to PostgreSQL it should be DBD/Pg.pm (upper case ‘P’, lower case ‘g’).

Missing dbi:driver: prefix

An error similar to:

Can't connect to data source 'bdi:Pg:dbname=finance;host=db.example.com' because I can't work out what driver to use (it doesn't seem to contain a 'dbi:driver:' prefix and the DBI_DRIVER env var is not set) at ./example.pl line 5

probably indicates an error in the data source passed to DBI::connect. In this case it is a typo: the prefix dbi has been erroneously replaced with bdi.

No pg_hba.conf entry for host

An error similar to:

DBI connect('dbname=finance;host=db.example.com','user',...) failed: FATAL:  no pg_hba.conf entry for host "192.168.0.2", user "user", database "finance", SSL off at ./example.pl line 6

indicates that the DBD module could not authenticate to the database server. There are several possible reasons why this could happen:

• the hostname is incorrect (but is running an instance of PostgreSQL);
• the name of the database is incorrect;
• the username is incorrect; or
• all of the above are correct but the database server has not been configured to accept them.

It does not indicate an incorrect password: PostgreSQL cannot and does not check the password until it has identified an authentication method that requires one, and in this case no such method was found.

Check that you can connect to the database using the psql command with the same set of credentials:

psql -d finance -h db.example.com -U user

If this fails then the problem relates to PostgreSQL (not to the DBI module) and will need to be fixed before you can proceed further. Otherwise, the most likely explanation is some difference between the credentials passed to connect and the credentials passed to psql.

Password authentication failed

If the password was wrong then this should be obvious from the error message:

DBI connect('dbname=finance;host=db.example.com','user',...) failed: FATAL:  password authentication failed for user "user" at ./example.pl line 5

Database does not exist

Similarly if the database does not exist:

DBI connect('dbname=finance;host=db.example.com','user',...) failed: FATAL:  database "finance" does not exist at ./example.pl line 5

See also

• Troubleshooting Perl DBI
• Connect to a database using Perl DBI

• Conventions
• Privacy
• Legal