E.103. Release 8.0

Support cross-data-type index usage (Tom)

Before this change, many queries would not use an index if the data types did not match exactly. This improvement makes index usage more intuitive and consistent.

New buffer replacement strategy that improves caching (Jan)

Prior releases used a least-recently-used (LRU) cache to keep recently referenced pages in memory. The LRU algorithm did not consider the number of times a specific cache entry was accessed, so large table scans could force out useful cache pages. The new cache algorithm uses four separate lists to track most recently used and most frequently used cache pages and dynamically optimize their replacement based on the work load. This should lead to much more efficient use of the shared buffer cache. Administrators who have tested shared buffer sizes in the past should retest with this new cache replacement policy.

Add subprocess to write dirty buffers periodically to reduce checkpoint writes (Jan)

In previous releases, the checkpoint process, which runs every few minutes, would write all dirty buffers to the operating system's buffer cache then flush all dirty operating system buffers to disk. This resulted in a periodic spike in disk usage that often hurt performance. The new code uses a background writer to trickle disk writes at a steady pace so checkpoints have far fewer dirty pages to write to disk. Also, the new code does not issue a global sync() call, but instead fsync()s just the files written since the last checkpoint. This should improve performance and minimize degradation during checkpoints.

Add ability to prolong vacuum to reduce performance impact (Jan)

On busy systems, VACUUM performs many I/O requests which can hurt performance for other users. This release allows you to slow down VACUUM to reduce its impact on other users, though this increases the total duration of VACUUM.

Improve B-tree index performance for duplicate keys (Dmitry Tkach, Tom)

This improves the way indexes are scanned when many duplicate values exist in the index.

Use dynamically-generated table size estimates while planning (Tom)

Formerly the planner estimated table sizes using the values seen by the last VACUUM or ANALYZE, both as to physical table size (number of pages) and number of rows. Now, the current physical table size is obtained from the kernel, and the number of rows is estimated by multiplying the table size by the row density (rows per page) seen by the last VACUUM or ANALYZE. This should produce more reliable estimates in cases where the table size has changed significantly since the last housekeeping command.

Improved index usage with OR clauses (Tom)

This allows the optimizer to use indexes in statements with many OR clauses that would not have been indexed in the past. It can also use multi-column indexes where the first column is specified and the second column is part of an OR clause.

Improve matching of partial index clauses (Tom)

The server is now smarter about using partial indexes in queries involving complex WHERE clauses.

Improve performance of the GEQO optimizer (Tom)

The GEQO optimizer is used to plan queries involving many tables (by default, twelve or more). This release speeds up the way queries are analyzed to decrease time spent in optimization.

Miscellaneous optimizer improvements

There is not room here to list all the minor improvements made, but numerous special cases work better than in prior releases.

Improve lookup speed for C functions (Tom)

This release uses a hash table to lookup information for dynamically loaded C functions. This improves their speed so they perform nearly as quickly as functions that are built into the server executable.

Add type-specific ANALYZE statistics capability (Mark Cave-Ayland)

This feature allows more flexibility in generating statistics for nonstandard data types.

ANALYZE now collects statistics for expression indexes (Tom)

Expression indexes (also called functional indexes) allow users to index not just columns but the results of expressions and function calls. With this release, the optimizer can gather and use statistics about the contents of expression indexes. This will greatly improve the quality of planning for queries in which an expression index is relevant.

New two-stage sampling method for ANALYZE (Manfred Koizar)

This gives better statistics when the density of valid rows is very different in different regions of a table.

Speed up TRUNCATE (Tom)

This buys back some of the performance loss observed in 7.4, while still keeping TRUNCATE transaction-safe.

Add WAL file archiving and point-in-time recovery (Simon Riggs)

Add tablespaces so admins can control disk layout (Gavin)

Add a built-in log rotation program (Andreas Pflug)

It is now possible to log server messages conveniently without relying on either syslog or an external log rotation program.

Add new read-only server configuration parameters to show server compile-time settings: block_size, integer_datetimes, max_function_args, max_identifier_length, max_index_keys (Joe)

Make quoting of sameuser, samegroup, and all remove special meaning of these terms in pg_hba.conf (Andrew)

Use clearer IPv6 name ::1/128 for localhost in default pg_hba.conf (Andrew)

Use CIDR format in pg_hba.conf examples (Andrew)

Rename server configuration parameters SortMem and VacuumMem to work_mem and maintenance_work_mem (Old names still supported) (Tom)

This change was made to clarify that bulk operations such as index and foreign key creation use maintenance_work_mem, while work_mem is for workspaces used during query execution.

Allow logging of session disconnections using server configuration log_disconnections (Andrew)

Add new server configuration parameter log_line_prefix to allow control of information emitted in each log line (Andrew)

Available information includes user name, database name, remote IP address, and session start time.

Remove server configuration parameters log_pid, log_timestamp, log_source_port; functionality superseded by log_line_prefix (Andrew)

Replace the virtual_host and tcpip_socket parameters with a unified listen_addresses parameter (Andrew, Tom)

virtual_host could only specify a single IP address to listen on. listen_addresses allows multiple addresses to be specified.

Listen on localhost by default, which eliminates the need for the -i postmaster switch in many scenarios (Andrew)

Listening on localhost (127.0.0.1) opens no new security holes but allows configurations like Windows and JDBC, which do not support local sockets, to work without special adjustments.

Remove syslog server configuration parameter, and add more logical log_destination variable to control log output location (Magnus)

Change server configuration parameter log_statement to take values all, mod, ddl, or none to select which queries are logged (Bruce)

This allows administrators to log only data definition changes or only data modification statements.

Some logging-related configuration parameters could formerly be adjusted by ordinary users, but only in the "more verbose" direction. They are now treated more strictly: only superusers can set them. However, a superuser can use ALTER USER to provide per-user settings of these values for non-superusers. Also, it is now possible for superusers to set values of superuser-only configuration parameters via PGOPTIONS.

Allow configuration files to be placed outside the data directory (mlw)

By default, configuration files are kept in the cluster's top directory. With this addition, configuration files can be placed outside the data directory, easing administration.

Plan prepared queries only when first executed so constants can be used for statistics (Oliver Jowett)

Prepared statements plan queries once and execute them many times. While prepared queries avoid the overhead of re-planning on each use, the quality of the plan suffers from not knowing the exact parameters to be used in the query. In this release, planning of unnamed prepared statements is delayed until the first execution, and the actual parameter values of that execution are used as optimization hints. This allows use of out-of-line parameter passing without incurring a performance penalty.

Allow DECLARE CURSOR to take parameters (Oliver Jowett)

It is now useful to issue DECLARE CURSOR in a Parse message with parameters. The parameter values sent at Bind time will be substituted into the execution of the cursor's query.

Fix hash joins and aggregates of inet and cidr data types (Tom)

Release 7.4 handled hashing of mixed inet and cidr values incorrectly. (This bug did not exist in prior releases because they wouldn't try to hash either data type.)

Make log_duration print only when log_statement prints the query (Ed L.)

Add savepoints (nested transactions) (Alvaro)

Unsupported isolation levels are now accepted and promoted to the nearest supported level (Peter)

The SQL specification states that if a database doesn't support a specific isolation level, it should use the next more restrictive level. This change complies with that recommendation.

Allow BEGIN WORK to specify transaction isolation levels like START TRANSACTION does (Bruce)

Fix table permission checking for cases in which rules generate a query type different from the originally submitted query (Tom)

Implement dollar quoting to simplify single-quote usage (Andrew, Tom, David Fetter)

In previous releases, because single quotes had to be used to quote a function's body, the use of single quotes inside the function text required use of two single quotes or other error-prone notations. With this release we add the ability to use "dollar quoting" to quote a block of text. The ability to use different quoting delimiters at different nesting levels greatly simplifies the task of quoting correctly, especially in complex functions. Dollar quoting can be used anywhere quoted text is needed.

Make CASE val WHEN compval1 THEN ... evaluate val only once (Tom)

CASE no longer evaluates the tested expression multiple times. This has benefits when the expression is complex or is volatile.

Test HAVING before computing target list of an aggregate query (Tom)

Fixes improper failure of cases such as SELECT SUM(win)/SUM(lose) ... GROUP BY ... HAVING SUM(lose) > 0. This should work but formerly could fail with divide-by-zero.

Replace max_expr_depth parameter with max_stack_depth parameter, measured in kilobytes of stack size (Tom)

This gives us a fairly bulletproof defense against crashing due to runaway recursive functions. Instead of measuring the depth of expression nesting, we now directly measure the size of the execution stack.

Allow arbitrary row expressions (Tom)

This release allows SQL expressions to contain arbitrary composite types, that is, row values. It also allows functions to more easily take rows as arguments and return row values.

Allow LIKE/ILIKE to be used as the operator in row and subselect comparisons (Fabien Coelho)

Avoid locale-specific case conversion of basic ASCII letters in identifiers and keywords (Tom)

This solves the "Turkish problem" with mangling of words containing I and i. Folding of characters outside the 7-bit-ASCII set is still locale-aware.

Improve syntax error reporting (Fabien, Tom)

Syntax error reports are more useful than before.

Change EXECUTE to return a completion tag matching the executed statement (Kris Jurka)

Previous releases return an EXECUTE tag for any EXECUTE call. In this release, the tag returned will reflect the command executed.

Avoid emitting NATURAL CROSS JOIN in rule listings (Tom)

Such a clause makes no logical sense, but in some cases the rule decompiler formerly produced this syntax.

Add COMMENT ON for casts, conversions, languages, operator classes, and large objects (Christopher)

Add new server configuration parameter default_with_oids to control whether tables are created with OIDs by default (Neil)

This allows administrators to control whether CREATE TABLE commands create tables with or without OID columns by default. (Note: the current factory default setting for default_with_oids is TRUE, but the default will become FALSE in future releases.)

Add WITH / WITHOUT OIDS clause to CREATE TABLE AS (Neil)

Allow ALTER TABLE DROP COLUMN to drop an OID column (ALTER TABLE SET WITHOUT OIDS still works) (Tom)

Allow composite types as table columns (Tom)

Allow ALTER ... ADD COLUMN with defaults and NOT NULL constraints; works per SQL spec (Rod)

It is now possible for ADD COLUMN to create a column that is not initially filled with NULLs, but with a specified default value.

Add ALTER COLUMN TYPE to change column's type (Rod)

It is now possible to alter a column's data type without dropping and re-adding the column.

Allow multiple ALTER actions in a single ALTER TABLE command (Rod)

This is particularly useful for ALTER commands that rewrite the table (which include ALTER COLUMN TYPE and ADD COLUMN with a default). By grouping ALTER commands together, the table need be rewritten only once.

Allow ALTER TABLE to add SERIAL columns (Tom)

This falls out from the new capability of specifying defaults for new columns.

Allow changing the owners of aggregates, conversions, databases, functions, operators, operator classes, schemas, types, and tablespaces (Christopher, Euler Taveira de Oliveira)

Previously this required modifying the system tables directly.

Allow temporary object creation to be limited to SECURITY DEFINER functions (Sean Chittenden)

Add ALTER TABLE ... SET WITHOUT CLUSTER (Christopher)

Prior to this release, there was no way to clear an auto-cluster specification except to modify the system tables.

Constraint/Index/SERIAL names are now table_column_type with numbers appended to guarantee uniqueness within the schema (Tom)

The SQL specification states that such names should be unique within a schema.

Add pg_get_serial_sequence() to return a SERIAL column's sequence name (Christopher)

This allows automated scripts to reliably find the SERIAL sequence name.

Warn when primary/foreign key data type mismatch requires costly lookup

New ALTER INDEX command to allow moving of indexes between tablespaces (Gavin)

Make ALTER TABLE OWNER change dependent sequence ownership too (Alvaro)

Allow CREATE SCHEMA to create triggers, indexes, and sequences (Neil)

Add ALSO keyword to CREATE RULE (Fabien Coelho)

This allows ALSO to be added to rule creation to contrast it with INSTEAD rules.

Add NOWAIT option to LOCK (Tatsuo)

This allows the LOCK command to fail if it would have to wait for the requested lock.

Allow COPY to read and write comma-separated-value (CSV) files (Andrew, Bruce)

Generate error if the COPY delimiter and NULL string conflict (Bruce)

GRANT/REVOKE behavior follows the SQL spec more closely

Avoid locking conflict between CREATE INDEX and CHECKPOINT (Tom)

In 7.3 and 7.4, a long-running B-tree index build could block concurrent CHECKPOINTs from completing, thereby causing WAL bloat because the WAL log could not be recycled.

Database-wide ANALYZE does not hold locks across tables (Tom)

This reduces the potential for deadlocks against other backends that want exclusive locks on tables. To get the benefit of this change, do not execute database-wide ANALYZE inside a transaction block (BEGIN block); it must be able to commit and start a new transaction for each table.

REINDEX does not exclusively lock the index's parent table anymore

The index itself is still exclusively locked, but readers of the table can continue if they are not using the particular index being rebuilt.

Erase MD5 user passwords when a user is renamed (Bruce)

PostgreSQL uses the user name as salt when encrypting passwords via MD5. When a user's name is changed, the salt will no longer match the stored MD5 password, so the stored password becomes useless. In this release a notice is generated and the password is cleared. A new password must then be assigned if the user is to be able to log in with a password.

New pg_ctl kill option for Windows (Andrew)

Windows does not have a kill command to send signals to backends so this capability was added to pg_ctl.

Information schema improvements

Add --pwfile option to initdb so the initial password can be set by GUI tools (Magnus)

Detect locale/encoding mismatch in initdb (Peter)

Add register command to pg_ctl to register Windows operating system service (Dave Page)

More complete support for composite types (row types) (Tom)

Composite values can be used in many places where only scalar values worked before.

Reject nonrectangular array values as erroneous (Joe)

Formerly, array_in would silently build a surprising result.

Overflow in integer arithmetic operations is now detected (Tom)

The arithmetic operators associated with the single-byte "char" data type have been removed.

Formerly, the parser would select these operators in many situations where an "unable to select an operator" error would be more appropriate, such as null * null. If you actually want to do arithmetic on a "char" column, you can cast it to integer explicitly.

Syntax checking of array input values considerably tightened up (Joe)

Junk that was previously allowed in odd places with odd results now causes an ERROR, for example, non-whitespace after the closing right brace.

Empty-string array element values must now be written as "", rather than writing nothing (Joe)

Formerly, both ways of writing an empty-string element value were allowed, but now a quoted empty string is required. The case where nothing at all appears will probably be considered to be a NULL element value in some future release.

Array element trailing whitespace is now ignored (Joe)

Formerly leading whitespace was ignored, but trailing whitespace between an element value and the delimiter or right brace was significant. Now trailing whitespace is also ignored.

Emit array values with explicit array bounds when lower bound is not one (Joe)

Accept YYYY-monthname-DD as a date string (Tom)

Make netmask and hostmask functions return maximum-length mask length (Tom)

Change factorial function to return numeric (Gavin)

Returning numeric allows the factorial function to work for a wider range of input values.

to_char/to_date() date conversion improvements (Kurt Roeckx, Fabien Coelho)

Make length() disregard trailing spaces in CHAR(n) (Gavin)

This change was made to improve consistency: trailing spaces are semantically insignificant in CHAR(n) data, so they should not be counted by length().

Warn about empty string being passed to OID/float4/float8 data types (Neil)

8.1 will throw an error instead.

Allow leading or trailing whitespace in int2/int4/int8/float4/float8 input routines (Neil)

Better support for IEEE Infinity and NaN values in float4/float8 (Neil)

These should now work on all platforms that support IEEE-compliant floating point arithmetic.

Add week option to date_trunc() (Robert Creager)

Fix to_char for 1 BC (previously it returned 1 AD) (Bruce)

Fix date_part(year) for BC dates (previously it returned one less than the correct year) (Bruce)

Fix date_part() to return the proper millennium and century (Fabien Coelho)

In previous versions, the century and millennium results had a wrong number and started in the wrong year, as compared to standard reckoning of such things.

Add ceiling() as an alias for ceil(), and power() as an alias for pow() for standards compliance (Neil)

Change ln(), log(), power(), and sqrt() to emit the correct SQLSTATE error codes for certain error conditions, as specified by SQL:2003 (Neil)

Add width_bucket() function as defined by SQL:2003 (Neil)

Add generate_series() functions to simplify working with numeric sets (Joe)

Fix upper/lower/initcap() functions to work with multibyte encodings (Tom)

Add boolean and bitwise integer AND/OR aggregates (Fabien Coelho)

New session information functions to return network addresses for client and server (Sean Chittenden)

Add function to determine the area of a closed path (Sean Chittenden)

Add function to send cancel request to other backends (Magnus)

Add interval plus datetime operators (Tom)

The reverse ordering, datetime plus interval, was already supported, but both are required by the SQL standard.

Casting an integer to BIT(N) selects the rightmost N bits of the integer (Tom)

In prior releases, the leftmost N bits were selected, but this was deemed unhelpful, not to mention inconsistent with casting from bit to int.

Require CIDR values to have all nonmasked bits be zero (Kevin Brintnall)

In READ COMMITTED serialization mode, volatile functions now see the results of concurrent transactions committed up to the beginning of each statement within the function, rather than up to the beginning of the interactive command that called the function.

Functions declared STABLE or IMMUTABLE always use the snapshot of the calling query, and therefore do not see the effects of actions taken after the calling query starts, whether in their own transaction or other transactions. Such a function must be read-only, too, meaning that it cannot use any SQL commands other than SELECT. There is a considerable performance gain from declaring a function STABLE or IMMUTABLE rather than VOLATILE.

Nondeferred AFTER triggers are now fired immediately after completion of the triggering query, rather than upon finishing the current interactive command. This makes a difference when the triggering query occurred within a function: the trigger is invoked before the function proceeds to its next operation. For example, if a function inserts a new row into a table, any nondeferred foreign key checks occur before proceeding with the function.

Allow function parameters to be declared with names (Dennis Björklund)

This allows better documentation of functions. Whether the names actually do anything depends on the specific function language being used.

Allow PL/pgSQL parameter names to be referenced in the function (Dennis Björklund)

This basically creates an automatic alias for each named parameter.

Do minimal syntax checking of PL/pgSQL functions at creation time (Tom)

This allows us to catch simple syntax errors sooner.

More support for composite types (row and record variables) in PL/pgSQL

For example, it now works to pass a rowtype variable to another function as a single variable.

Default values for PL/pgSQL variables can now reference previously declared variables

Improve parsing of PL/pgSQL FOR loops (Tom)

Parsing is now driven by presence of ".." rather than data type of FOR variable. This makes no difference for correct functions, but should result in more understandable error messages when a mistake is made.

Major overhaul of PL/Perl server-side language (Command Prompt, Andrew Dunstan)

In PL/Tcl, SPI commands are now run in subtransactions. If an error occurs, the subtransaction is cleaned up and the error is reported as an ordinary Tcl error, which can be trapped with catch. Formerly, it was not possible to catch such errors.

Accept ELSEIF in PL/pgSQL (Neil)

Previously PL/pgSQL only allowed ELSIF, but many people are accustomed to spelling this keyword ELSEIF.

Improve psql information display about database objects (Christopher)

Allow psql to display group membership in \du and \dg (Markus Bertheau)

Prevent psql \dn from showing temporary schemas (Bruce)

Allow psql to handle tilde user expansion for file names (Zach Irmen)

Allow psql to display fancy prompts, including color, via readline (Reece Hart, Chet Ramey)

Make psql \copy match COPY command syntax fully (Tom)

Show the location of syntax errors (Fabien Coelho, Tom)

Add CLUSTER information to psql \d display (Bruce)

Change psql \copy stdin/stdout to read from command input/output (Bruce)

Add pstdin/pstdout to read from psql's stdin/stdout (Mark Feit)

Add global psql configuration file, psqlrc.sample (Bruce)

This allows a central file where global psql startup commands can be stored.

Have psql \d+ indicate if the table has an OID column (Neil)

On Windows, use binary mode in psql when reading files so control-Z is not seen as end-of-file

Have \dn+ show permissions and description for schemas (Dennis Björklund)

Improve tab completion support (Stefan Kaltenbrunn, Greg Sabino Mullane)

Allow boolean settings to be set using upper or lower case (Michael Paesold)

Use dependency information to improve the reliability of pg_dump (Tom)

This should solve the longstanding problems with related objects sometimes being dumped in the wrong order.

Have pg_dump output objects in alphabetical order if possible (Tom)

This should make it easier to identify changes between dump files.

Allow pg_restore to ignore some SQL errors (Fabien Coelho)

This makes pg_restore's behavior similar to the results of feeding a pg_dump output script to psql. In most cases, ignoring errors and plowing ahead is the most useful thing to do. Also added was a pg_restore option to give the old behavior of exiting on an error.

pg_restore -l display now includes objects' schema names

New begin/end markers in pg_dump text output (Bruce)

Add start/stop times for pg_dump/pg_dumpall in verbose mode (Bruce)

Allow most pg_dump options in pg_dumpall (Christopher)

Have pg_dump use ALTER OWNER rather than SET SESSION AUTHORIZATION by default (Christopher)

Make libpq's SIGPIPE handling thread-safe (Bruce)

Add PQmbdsplen() which returns the display length of a character (Tatsuo)

Add thread locking to SSL and Kerberos connections (Manfred Spraul)

Allow PQoidValue(), PQcmdTuples(), and PQoidStatus() to work on EXECUTE commands (Neil)

Add PQserverVersion() to provide more convenient access to the server version number (Greg Sabino Mullane)

Add PQprepare/PQsendPrepared() functions to support preparing statements without necessarily specifying the data types of their parameters (Abhijit Menon-Sen)

Many ECPG improvements, including SET DESCRIPTOR (Michael)

Allow the database server to run natively on Windows (Claudio, Magnus, Andrew)

Shell script commands converted to C versions for Windows support (Andrew)

Create an extension makefile framework (Fabien Coelho, Peter)

This simplifies the task of building extensions outside the original source tree.

Support relocatable installations (Bruce)

Directory paths for installed files (such as the /share directory) are now computed relative to the actual location of the executables, so that an installation tree can be moved to another place without reconfiguring and rebuilding.

Use --with-docdir to choose installation location of documentation; also allow --infodir (Peter)

Add --without-docdir to prevent installation of documentation (Peter)

Upgrade to DocBook V4.2 SGML (Peter)

New PostgreSQL CVS tag (Marc)

This was done to make it easier for organizations to manage their own copies of the PostgreSQL CVS repository. File version stamps from the master repository will not get munged by checking into or out of a copied repository.

Clarify locking code (Manfred Koizar)

Buffer manager cleanup (Neil)

Decouple platform tests from CPU spinlock code (Bruce, Tom)

Add inlined test-and-set code on PA-RISC for gcc (ViSolve, Tom)

Improve i386 spinlock code (Manfred Spraul)

Clean up spinlock assembly code to avoid warnings from newer gcc releases (Tom)

Remove JDBC from source tree; now a separate project

Remove the libpgtcl client interface; now a separate project

More accurately estimate memory and file descriptor usage (Tom)

Improvements to the Mac OS X startup scripts (Ray A.)

New fsync() test program (Bruce)

Major documentation improvements (Neil, Peter)

Remove pg_encoding; not needed anymore

Remove pg_id; not needed anymore

Remove initlocation; not needed anymore

Auto-detect thread flags (no more manual testing) (Bruce)

Use Olson's public domain timezone library (Magnus)

With threading enabled, use thread flags on Unixware for backend executables too (Bruce)

Unixware cannot mix threaded and nonthreaded object files in the same executable, so everything must be compiled as threaded.

psql now uses a flex-generated lexical analyzer to process command strings

Reimplement the linked list data structure used throughout the backend (Neil)

This improves performance by allowing list append and length operations to be more efficient.

Allow dynamically loaded modules to create their own server configuration parameters (Thomas Hallgren)

New Brazilian version of FAQ (Euler Taveira de Oliveira)

Add French FAQ (Guillaume Lelarge)

New pgevent for Windows logging

Make libpq and ECPG build as proper shared libraries on OS X (Tom)

Overhaul of contrib/dblink (Joe)

contrib/dbmirror improvements (Steven Singer)

New contrib/xml2 (John Gray, Torchbox)

Updated contrib/mysql

New version of contrib/btree_gist (Teodor)

New contrib/trgm, trigram matching for PostgreSQL (Teodor)

Many contrib/tsearch2 improvements (Teodor)

Add double metaphone to contrib/fuzzystrmatch (Andrew)

Allow contrib/pg_autovacuum to run as a Windows service (Dave Page)

Add functions to contrib/dbsize (Andreas Pflug)

Removed contrib/pg_logger: obsoleted by integrated logging subprocess

Removed contrib/rserv: obsoleted by various separate projects