You are on page 1of 125

1

Migration Guide to Firebird 3


First edition – 2016
Author: Carlos Henrique Cantu
Piracicaba – São Paulo – Brazil

Copyright 2016 © Carlos Henrique Cantu


All rights reserved by the author

No part of this book may be reproduced, shared, transmitted


and/or recorded by any means, without prior written permission
from the author.

Editing, translation, diagramming, finalization:


Carlos Henrique Cantu

Proofreading: Ann Harrison

Revision 1.02

Pages changed in revision 1.02: 46, 59, 65, 70, 90, 93 and 112.

Index
2

Index
Index............................................................................................ 2
Dedication ................................................................................... 5
Thanks ......................................................................................... 6
About the author........................................................................ 11
Preface ....................................................................................... 12
Introduction ............................................................................... 14
Icons used .................................................................................. 15
Errata ......................................................................................... 16
Basic but essential concepts!..................................................... 17
SuperServer vs. Classic vs. SuperClassic .............................. 18
Classic (CS)....................................................................... 20
SuperServer (SS) ............................................................... 21
SuperClassic (SC) ............................................................. 22
Embedded .......................................................................... 22
What architecture to choose? ................................................ 23
32 vs. 64 bits.......................................................................... 25
Installing Firebird 3 ................................................................... 27
Installing Firebird 3 on Linux ............................................... 28
Installing Firebird on Windows® ......................................... 35
Server architecture ............................................................ 40
Service or Application? ..................................................... 40
Start automatically ............................................................ 40
Client library (fbclient.dll) ................................................ 40
gds32.dll ............................................................................ 41
Authorization for legacy Firebird clients .......................... 41
Checking whether Firebird is running............................... 43
Installing Firebird using the “Zip Kit” .............................. 46
INSTSVC .......................................................................... 46
INSTREG .......................................................................... 47
INSTCLIENT.................................................................... 48
Migrating Existing Databases to Firebird 3 .............................. 49
Why Migration? .................................................................... 50
ODS (On Disk Structure) ...................................................... 51
Test the database integrity with gbak .................................... 52
Problems with character encoding ........................................ 54

Index
3

Validating the metadata......................................................... 55


Migrating a database to Firebird 3 ........................................ 59
Migrating 24x7 servers ......................................................... 60
Tips to speed up the backup/restore process ........................ 61
Users in Firebird 3..................................................................... 62
Local users ............................................................................ 63
Passwords .............................................................................. 65
Initializing the security database ........................................... 67
Managing users using SQL ................................................... 68
Creating users .................................................................... 69
Modifying users ................................................................ 71
Deleting users .................................................................... 71
Sec$users and sec$user_attributes virtual tables................... 72
Preparing a script to insert users into the new server ............ 74
Protecting your data .................................................................. 79
Creating a secure environment .............................................. 81
Encrypting the database file .................................................. 82
Conclusion............................................................................. 84
Wire Protocol Enhancements .................................................... 85
Traffic encryption.................................................................. 86
Traffic compression .............................................................. 88
Enhancements for usage in high latency networks ............... 90
Connection strings .................................................................... 95
Legacy syntax........................................................................ 96
URL based syntax ................................................................. 98
IPv6 support ........................................................................ 101
Firebird 3 and legacy applications .......................................... 102
.NET applications................................................................ 103
Jaybird applications............................................................. 104
Logical data type (Boolean) ................................................ 105
Connecting to Firebird 3 with an old client library (fbclient)
............................................................................................... 105
Query performance.............................................................. 105
Reserved words ................................................................... 106
Manipulating the System tables (RDB$...) ......................... 108
Testing application’s queries............................................... 110
Using mon$attachments to get the number of active
connections............................................................................ 111

Index
4

Default cache size for Classic/SuperClassic ....................... 112


Mixing implicit and explicit joins ....................................... 112
Count() now returns a BIGINT ........................................... 113
Appendix ................................................................................. 114
Macros ................................................................................. 115
Configuration entries........................................................... 116
Glossary................................................................................... 118

Index
5

Dedication

Dedicated to my wife Elizabeth, my daughter Larissa, and to my


parents, Carlos and Sônia.

Carlos Henrique Cantu

Dedication
6

Thanks
I want to thank everyone who participated in the Migration
Guide’s crowd funding campaign! Thank you for your confidence!
Choosing Firebird as the main (and the only) RDBMS used in
my applications, put me in contact with people from all over the
world, whether through emails, chats or even personally, in the
international Firebird conferences and in the Firebird Developers
Days. Many of these people ended up becoming great colleagues,
and many of them were an important part solving questions during
the writing process of this Migration Guide.
Many thanks to the core-developers Vlad Khorsun, Alex Peshkov
and Dmitry Yemanov, who were always prompt to answer my doubts.
Jiří Činčura and Mark Rotteveel for answering questions about the
.NET Provider and Jaybird issues, Paul Reeves, responsible for the
Firebird installer for Windows, Alexey Kovyazin and Dmitry Kuzmenko,
for the long-standing partnership that turned out to be a strong
friendship, and Ivan Prenosil, for the long conversations on common
interests, which often went far beyond Firebird.
Obviously, Ann Harrison, by the constant sympathy and for
always being willing to help (even with guitars and video-games )
as well for the awesome proofreading work in this book, and her
husband, Jim Starkey, for creating InterBase© and thus made Firebird
possible. Jederson Zuchi and Marcelo Daibert for helping with the
review of the Portuguese version of the book and, finally, Adriano
dos Santos Fernandes, for being the only Brazilian actively
participating in the Firebird development.

Carlos Henrique Cantu


www.firebase.com.br / blog.firebase.com.br / www.firebirdnews.org
www.supercontrol.com.br / www.warmboot.com.br

Thanks
7

Below is the list of people/companies who participated in the


book’s crowdfunding campaign:

Acesys Tecnologia em Adilson Pazzini


Sistemas Ltda
Adriano Wolff Alberto Ricardo Gonçalves de Souza
Alex Antunes da Silva Alexandre Chaves Martins
Alexandre de Souza Pinheiro Alexandre G. Camilo
Alirio Botelho Junior Aloisio Sampaio Cairo
Americo Belmiro de Souza Amilcar Coelho de
Neto Vasconcellos
André Fernando Piasentin Andrei Luis dos Santos
Angelo Ricardo Miquelin Neto Anônimo
Anselmo Souza Antonio Carlos de Moura
Antonio Carlos Nunes Junior Ap Engenharia de Software
Aparecido dos Santos Silva Ari Rodrigues da Silva
Ari Tadashi Sato Arielton Lima
Artur Bezerra Artur Moreira
Ascont Sistemas Beneval A. P. Junior

Carlos Alberto Santa Clara Carlos Antonio Morcerf


Carlos do Nascimento Filho Carlos E. Tré
Carlos Eleutério dos Reis Carlos Luiz Francisco
Carlos Rempto Carsoft Sistemas
Casasoft desenvolvimento de CF Company Desenvolvimento de Software

Sistemas Ltda
Chrystian Sweick Silva Cigo Software de Gestao
Claudio Brasileiro Pimenta Cleverson Ruivo da Silva
Cristiano Schenatto D&O Sistemas
Daisson André dos Santos Daniel Alves Qualhato

Daniel Simões de Almeida Daniel Yamamoto


Darcy Luiz Muller Daré, Gimenez e Cia Ltda Me
Dari Junior - DSoft Solutions data C Informática
Delcy Dias Diassis Bonfim Macedo
Diego Fernandes de Souza Dimas Fernando Braga
Dirceu Altair Henchen Dorival da Silva Reis
Dream Sistemas Dsin - Tecnologia da
Informação
Eder Laudelino Polizel Eder Paz
Edgar da Silva Oliveira Edison Kemerich de Brito
Edivaldo Venancio da Silva Edmar de Oliveira Frazão
Eduardo Brigoni Eduardo Sório Neto

Thanks
8

Eduardo Suruagy Eduardo Trevisan


Eldon Rodrigues Aquino Eney Duarte Gomes - Versátil
Informática
Erick A M P
Erick Figueira
Erik Fabiano Elias Fabiano de Oliveira Prado
Fábio Albano Fábio Henrique Guimarães
Fábio Henrique Pereira do Fabio José Pandim
Nascimento
Fabrício Carvalho de Matos Fernando Oliveira Pereira
Fernando Pimenta Fernando Quartarollo
Flavio Hermes Francisco Carlos Rodrigues
Francisco Nilson Brasil da Costa Francisco Rogeres Sousa de Melo
Francisco Schimitt
Fusiontech® Sistemas
Genésio Becker George De Luca
Gerasoft Informática Ltda. Giga Informática Dracena Ltda -
Me
Gilberto S Moura Gilson Inacio do Carmo
Gilson Peloso Gleisson Leal
Grupo Acert Guto (Gutemberg Lima Sampaio)
Hélio José Almeida de Oliveira Helton Alves Costa
Hercules Gomes Cangussu Hugo Eyng
Hugo Fabrício Gonçalves e Silva Humberto Mendes
Hyperbyte Informática IBS Sistemas
Idez Sistemas Infomatte Sistemas e
Consultoria
Ismael da Silva Marques Ivan José de Mecenas Silva
Jacson Vilson Meyer Jardel Thomas
Jederson donizete Zuchi Jéter Rabelo Ferreira
Joao Batista Dias da Cruz João Bosco dos Santos
Joao Carlos S. Santos João Marcelo Collar
Joao Rodrigues dos Santos Jorge Rocha Filho
José Alberto Ferrão Silva José Antonio Bonna
José B.Moreira Jose Bezerra
Jose Helder Camurca Junior José Henrique Godoi Alves
José Nazareno Freire Neto Jose Nilton Pace
Jose Pereira de Souza Júlio César Andrade dos Anjos
Julio Cesar Mazoni Kelver Merlotti
Keysolution Kk Borges BH

Kleberson Toro Vidal Leandro Bertasso

Leandro Irgang Leonardo Wascheck –


Intercamp Sistemas S.A.

Thanks
9

Limber Software(Ronaldo Loercio Luiz Relly


Bett e Fernando Fiorentin)
Loercio Luiz Relly Luciano Carneiro dos Santos
Luis Paulo Alves Rodrigues Luis Roberto Vieira
Luis Wagner dos Santos Luis Zen
Luiz Henrique Spies Luiz C. G. Mota
Luiz Filipe Meinecke Marcello Cainelli
Marcelo Augusto Cicogna Marcelo Duraes da Silva
Marcelo Estanislau Geyer Marcelo Rodrigues de Oliveira
Marcelo Silva Márcio Alves dos Santos
Marcio Bertelli Marcos Fábio Travain
Marcos Pegoraro Marcos Torres
Mario Aldo Serafim Inacio Mario Hollanda
Marlene Abi-Ali Mauri Lima de Oliveira
Mauricio Junqueira Mauro Gomes
Melkcimar Francisco da Costa Menandro Dias Evangelista Neto
Miari Tecnologia Ltda Miguel Angel Romero Arcas
Nelson A. Servija Vechini Nelson Henrique Corrêa
Nepomuceno
Nelson Vilas Boas Nirlan E. Fundão
Orsini Vicari Aguiar Oscar Luiz Rodrigues de Oliveira
Osvaldo Goncalves da Cruz Júnior Otavio Benini - Benini
Informática E Sistemas
Otávio Ferdin Júnior Paulo Eduardo Strahler
Paulo Henrique Albanez Paulo Pereira Botelho
(PHA)
Paulo Roberto Lucas Lázaro Paulo Santana
Pedro Santo André Neto Prevedello Sistemas Ltda
Primesoftware Rafael Moreira Neves
Rc Equipe Em Informática Com Reginaldo José Fiomano
E Serv Ltda
Renato Miranda Renato Piovesano Bartolmei
Ricardo da Silva Almeida Ricardo Faria Beatriz
Robert de Godoy Barbosa Roberto Couto dos Santos
Roberto Keri Robson Ferreira Gomes
Rodrigo Belmiro Rodrigo Mourão
Rogerio deckert Zek Rondinely de Souza Soares
Roubledo demiam Gasoni Rubens Moraes Santos
Rudar Informatica Ltda Rui Richard Blaese
Saac.Software Sady Moreira
Samuel Santana dos Santos Saul Godino da Silva Filho

Thanks
10

Sergio de Siqueira Silbeck Sistemas


Siro Costa Marques
Staff Consultoria Em
Informática Ltda
Sysbase Informática Ltda Syscon Tecnologia - Raitecsul
Syslife Sistemas Systemar Informática Ltda
Technosystem Assessoria E Sistemas Teodorico Nicolau Rodrigues Junior
Tham Informática Ltda Thiago Falcao Pereira
Valdemir Jacon Sanches Valdenir Rigonatto
Valdir Stiebe Junior Vinicius de Souza Barreira
Virtua Software Ltda VirtualSoft Informática e
Tecnologia Ltda
Wagner Machado Walfrido Cesar Cintra
Wander Pereira dos Santos Wanderson Pereira
Webdutos Wellington Rodrigo Sala
Whanderson Santos Rodrigues Wilton de Souza Magalhaes
Youngarts Sistemas

Thank you all once again!

Thanks
11

About the author


Carlos Henrique Cantu is well-known in the Brazilian Firebird
users' community, having launched the first Portuguese speaking
portal devoted to Firebird, even before it existed (in the InterBase©
days). FireBase (www.firebase.com.br) has more than 70,000
subscribed users, and offers free access to Firebird articles, FAQ,
discussion list, consulting services, downloads as well an on-line
store.
Carlos has published dozens of articles in several programming
magazines in Brazil, was the president of DUG-BR (Delphi Users
Group Brazil), editor of the DB FreeMagazine, columnist of the Active
Delphi magazine and organizer of the Firebird Developers Day
conference - the biggest Firebird conference in the world - with an
average of 450 attendees each year, and currently in its 13th year. He
also published two books about Firebird (Firebird Essencial and
Firebird 2 – both of them available only in Portuguese), spoke at five
different International Firebird Conferences, at every Brazilian DDD
(Delphi Developers Day) and FDD (Firebird Developers Day), and several
other seminars about databases and programming.

About the author


12

Preface
Firebird 3.0 is the most significant new release since Firebird's
appearance in 2000. New features range from finely grained multi-
threading that allows SuperServer to exploit symmetric multi-
processors to a new security architecture that allows authentication
within a database, to encrypted communications.
Old limits on the useful length of passwords, database size, and
the total number of transactions that can be executed on a database
have been raised. Interface changes include a new object-oriented
API, provision for user-written plug-ins to extend Firebird's
capabilities, and a provider interface that simplifies engine
development and reduces the complexity of shifting between
embedded and server-based access.
Firebird 3 is faster, more secure, more capable, and more flexible
than any previous version.
Carlos Cantu has done the Firebird community a great service by
explaining how to move existing applications from earlier versions
of Firebird to Firebird 3 in clear and precise terms. Carlos has been
active with Firebird since its beginning. He has written two books
and dozens of articles about Firebird over the years. He has been
instrumental in supporting and encouraging the use of Firebird in
Brazil through the FireBase portal. When he decided to write a
migration guide to help Firebird users move their databases to
Firebird 3, more than 200 companies and individual developers
stepped up to help fund the effort. Firebird 3 is an enormous
release! Breaking the new features into groups is the only way to
understand and use them. The right place to start is explaining how
to move applications and databases from earlier Firebird versions to
Firebird 3.
Unfortunately, for the larger Firebird community, Carlos's earlier
books are available only in Portuguese, so his clear, step-by-step
instructions for getting the most from Firebird applications reached
only a small part of their potential audience. He translated this book
to English, a language available to most software developers.

Preface
13

The Migration Guide does not replace the release notes that come
with Firebird. Instead, it expands and explains a small part of what
the release notes cover, in a way that reduces the potential for
confusion and errors in moving to Firebird 3.
This book is what you need for a successful start with Firebird 3.

Ann Harrison
April/2016

Preface
14

Introduction
After a hiatus of almost 10 years, since the release of my last
book (Firebird 2), I felt it was time to write something new. After
some thinking, I decide to take the opportunity of the release of the
long-awaited Firebird 3, and help those who want to migrate their
existing servers to this new version.
The story of Firebird 3 could easily fit in a Mexican novel!
Numerous delays, broken promises, date changes, etc. made some
people doubt that Firebird 3 would ever be released! Fortunately, it's
about to happen or, perhaps, has already happened - depending on
when you read this book!
Firebird 3 brings numerous innovations, such as the long-awaited
full SuperServer’s SMP support, network and database encryption,
local user authentication in the database, improvements in the
communication protocol, in addition to several new features in
different areas of the DBMS. All this made the migration process
from an older Firebird version a bit more complicated than it was in
previous versions, where, basically, all you had to do was replace the
server with the new Firebird version or, at worst, a backup and
restore of the database. This was why I chose to write a migration
guide!
The book covers only the essential topics related to the migration
process. To stay up to date with all the news regarding Firebird 3,
including new PSQL commands, it is really important that you
read the Release Notes!
I recommend to all the readers to access www.firebirdnews.org
periodically, to stay informed about what is new in the Firebird
world.
I hope you have a pleasant reading!

Carlos H. Cantu
April/2016

Introduction
15

Icons used
In this book, you'll find several icons pointing to subjects that
deserves special attention.

 Ignoring the subject may put you in a situation of


“imminent danger”!

 Stop and read carefully!

 Ignoring the subject can lead to a dangerous


situation which can compromise the availability
of the server and the environment.

 Important / Tip

In the end of the book, there is a glossary with the definition of


several terms and abbreviations found in Firebird literature.

Icons used
16

Errata
Errors and mistakes that may be discovered after the publication
of this book will be listed in www.firebirdnews.org/migration-guide-
to-firebird-3/ along with the corrections.
I advise visiting the page periodically.

Errata
17

Basic but essential


concepts!
Don’t even think about skipping this chapter!

Basic but essential concepts!


18

Before turning to the new features of Firebird 3, you need to


understand a few important concepts about the way Firebird works
and the reasons for some of the changes.
Firebird can run in three different "architectures" (modes).
Understanding those modes can be the difference between success
and frustration during its use.

SuperServer vs. Classic vs. SuperClassic

Firebird can run in any one of three different architectures


(modes): Classic, SuperServer, and SuperClassic. To understand the
difference between them, you need to understand a bit of the
internal workings of Firebird.
A Firebird database file is composed of "logical blocks" called
pages. Pages are all the same size. The page size for a database is
defined at database creation or when restoring a backup created by
gbak using the -p(age) parameter. Each page in a Firebird database
has a specific use: some hold data, some hold segments of an index,
others are transaction inventory pages, or blob pages, or pages of
sequences and others manage internal structures.
When a Firebird accesses a database, it allocates a certain amount
of RAM as a cache, to speed up the access to pages recently read
from the database file. Firebird has two caches for each active
database: a Metadata cache and a Page cache. The Metadata cache, as the
name suggests, holds the metadata of the database, the definitions
of all the objects that compose the database (tables, procedures,
triggers, etc.). The Page cache holds pages that were read recently, so a
second reference to a page is much faster than the first, since there
is no need to retrieve it from the disk. When Firebird needs the
information from a page of the database file, it first checks whether
that page is already in its own cache. If not, it requests that page
from the operating system. The operating system checks for the page in its
own disk cache. If the operating system does not find the page, it asks its
storage system to retrieve the page.
In general, we can imagine different cache layers, as shown in the
following picture:

Basic but essential concepts!


19

Firebird page cache

Operating System cache

Storage device cache

Database file

The amount of memory allocated for Firebird’s page cache can be


set for all databases on a system in the firebird.conf file, or for a single
database using gfix -buffers parameter. When the page cache size is set
for a particular database, the cache’s size is written to the header of
the database file. The size of the cache (buffer) is always set as a
number of pages, not bytes. The number of buffers multiplied by the
database's page size determines the total amount of memory allocated
by Firebird for its page cache. For example, in a database with 8 KB
pages, setting the buffers to 4,000 allocates 32,768,000 bytes of
memory (4,000 x 8,192 bytes) = ~32 MB.

 When the buffers value stored in the header of the


database is zero, Firebird uses the default global
value set in the parameter DefaultDbCachePages in
firebird.conf.

The use of the page cache is the first difference between the
Firebird architectures. In the SuperServer mode, all connections to
a database share the page cache among themselves. In the Classic
and SuperClassic versions, each connection has its own cache,
which is not shared. This subject is described in more detail later
in this chapter.
A second difference between Firebird operating modes is how
connections are managed: by threads or by processes and whether
requests from processes can be executed in parallel.

Basic but essential concepts!


20

Here are the characteristics of the three Firebird architectures,


how they differ, and the advantages of each.

Classic (CS)

In the Classic version, each database connection is served by a


separate Firebird process. There are as many Firebird processes
running as there are active connections. Each process has a database
cache. The caches are not shared among them.
In the previous example, if we had 10 connections accessing the
same database, the total memory allocated for the page cache in
Firebird Classic would be ~320 MB (10 x 32 MB).

 One frequent cause of performance degradation


when using the Classic or SuperClassic architectures
is setting a very large buffer size. If the page caches
allocated by connections exceed the size of the
system's available physical memory, the operating
system uses virtual memory, generating disk swaps,
and degrading the server performance as a whole.

Since each connection is a separate Firebird process, connections


in Classic are quite independent from each other. One process can
be killed without affecting other connections. When a connection
overloads the server, you can kill the Firebird process for that
connection. All other connections continue to operate normally.
Independent processes reduce the damage done when a connection
has a fatal error, like using a buggy UDF (User Defined Function), and
crashes. With Classic, only the Firebird process that called the UDF
crashes: others continue untroubled.

 A rare problem can occur when using Classic on


Windows. Serving many connections generates as
many processes and can cause Windows to hit its
threshold for desktop heap. To solve that problem,
change the Windows user associated with the
Firebird process from LocalSystem to something
else.

Basic but essential concepts!


21

In Classic, each Firebird process consumes server resources, not


just the memory used for cache. Proper configuration of the
hardware “power” and the operating system can avoid exhausting
system resources. Using a 64-bit operating system and Firebird
version can also avoid resource problems.
On Windows, a process called fb_inet_server.exe for earlier
versions of Firebird, and firebird.exe for Firebird 3 is always present,
even when there are no active connections. This process is a listener.
When a new connection request arrives, the listener receives the
request and starts a new Firebird process to serve that connection.
On Linux, usually xinetd is the listener.

SuperServer (SS)

In the SuperServer architecture, connections are served by threads


of a single Firebird process. The database page cache is shared among
all the connections to a database. The shared cache reduces memory
consumption and I/O load. Pages from the cache do not require
synchronization of caches as they do in Classic and SuperClassic.
However, having a single process serve many connections
reduces the independence of connections. If any connection
overloads the server, killing the Firebird process kills all
connections.
Another disadvantage is that, prior to version 2.5, SuperServer
could not take advantage of parallel execution on multi-core
machines (SMP). On Windows, Firebird must be attached to a
single processor/core to avoid having the operating system try to
balance usage by shifting the process from one core to another.
Fortunately, Firebird 3 solves this limitation!

 In Firebird 2.5, SS can use more than one core,


but only when accessing multiple databases on a
server. To do this, you must configure the
CPUAffinitymask parameter of the firebird.conf file in
a way that allows the use of two or more
processors. However, Firebird 2.5 can use only
one core per database.

Basic but essential concepts!


22

SuperClassic (SC)

The SuperClassic is a hybrid of the Classic and SuperServer. Like


SuperServer, one SuperClassic Firebird process serves all connections
using threads. However, the cache works like Classic, so each
connection has a private page cache.
SuperClassic scales better than the SuperServer (in Firebird 2.5) and
Classic. The synchronization of pages between caches is more
efficient than Classic, since there is no inter process communication
overhead. A single process synchronizes access to database pages.
Because SuperClassic allocates a page cache for each connection,
the same rule for calculating cache’s RAM consumption used in
Classic should be applied for SuperClassic to avoid excessive
consumption and/or exhaustion of the server’s RAM.
As the SuperClassic has only one process, it consumes fewer
server resources (like process space) compared to the Classic. The
SuperClassic architecture was a transitional step to Firebird 3, in
which SuperServer runs connection threads in parallel on multi-core
servers.

Embedded

The embedded version allows Firebird to be distributed with


applications, without any separate installation step, making it simple
to distribute demo versions of software, digital catalogs, etc. In
Firebird 3, Embedded is not a separate architecture but a limited and
simplified version of Firebird that can support any of the three
architectures.
Before Firebird 3, the embedded version of Firebird was
distributed as a shared library called fbembed.dll that had the same
interface as the client library, fbclient.dll, but included all the functions
of the Firebird server. Renaming the file fbembed.dll to fbclient.dll
caused an application to load the whole server for local use!
In Firebird 3, application developers can distribute their
application along with the “standard” fbclient.dll plus some libraries
and support files that provide the functions of the RDBMS. The
application connects the database providing only the local file name

Basic but essential concepts!


23

or alias, but not a remote address. As a result, you can configure the
embedded server to act as a SuperServer, Classic, or SuperClassic.
These files must be distributed with applications that use Firebird
in embedded mode:
fbclient.dll - client libary
firebird.msg - message file
ib_util.dll - memory allocation for UDFs that return data to Firebird
icudt52.dll - internationalization package
icudt52l.dat - internationalization package
icuin52.dll - internationalization package
icuuc52.dll - internationalization package
firebird.conf – if you need to use some non default configurations
databases.conf – if you wanna to use alias or define per DB config.
msvcp100.dll – if the MSVC runtime is not yet available on Windows
msvcr100.dll – if the MSVC runtime is not yet available on Windows
intl\fbintl.conf – if you use charset different to NONE
intl\fbintl.dll – if you use charset different to NONE
plugins\engine12.dll - server code

Firebird embedded does not authenticate users logging into the


database. The login simply ignores the password. You can restrict
user access to certain database objects using the grant/revoke
command.
Following is a summary of the differences among the available
architectures in Firebird 3.

Firebird 3 Executable SMP Shared File open in Threads/


Architecture name friendly cached exclusive Process
mode
SuperServer firebird YES Yes Yes Threads
Classic firebird Yes No No Process
SuperClassic firebird Yes No No Threads

What architecture to choose?

There are no hard and fast rules or "recipes" for choosing the
best architecture for every imaginable situation. Many variables and
special needs influence the choice of the most suitable architecture
for each installation.
However, here is a "general guide" that can serve as the basis for
choosing among the architectures. Your needs may not fit in this
"recipe". The recipe assumes that the hardware and operating
system of the server are configured correctly.

Basic but essential concepts!


24

Classic – Can be used on multiprocessor servers, supporting a


large number of concurrent connections. It is "SMP" friendly. The
operating system distributes Firebird processes among all available
cores. Classic is especially interesting when the application requires
maximum stability, because one failed Firebird process does not
affect other connections.

SuperClassic – Can be used in multiprocessor servers - it is


SMP friendly - but it should be used only on 64-bit operating
systems with a 64-bit Firebird version. Supports a large number of
concurrent connections and consumes fewer operating system
resources than Classic. SuperClassic scales better than Classic because
cache synchronization does not require communication among
processes. However, unlike Classic, the crash of a Firebird
connection kills all the existing connections.

The SuperServer architecture deserves a special attention, because


it behaves differently in Firebird 3 than in older versions. Let's see:

SuperServer in Firebird up to 2.5 – Can be used by those who


have little or no experience with Firebird or relational databases.
Should be used only when there are few concurrent connections,
when queries performed on the database are short and optimized,
and when the shared cache among connections justifies the choice
despite the lack of SMP support.

SuperServer in Firebird 3.0 –SuperServer is the standard


architecture in Firebird 3. Firebird 3 SuperServer is "SMP" friendly;
it takes advantage of all CPUs/cores available in the server machine,
and still offers a shared page cache – the best of both worlds!

 Dmitry Yemanov, head of the Firebird


development team, says in his Performance
Optimization MasterClass that "unless you're
paranoid about stability, use SuperServer". In other
words: if the option to "kill" a Firebird process
without affecting the other connections is really
important for you (paranoia?), then use Classic.

Basic but essential concepts!


25

SuperClassic is still supported, but has no


advantages over SuperServer.

In Firebird 3, the desired architecture (CS, SS or SC) is set for all


databases on a server with the new ServerMode parameter, in
firebird.conf. In the future, this parameter may be controllable on
individual databases through the databases.conf file.
The possible values for ServerMode are:

• Super (or ThreadedDedicated) – is the standard


architecture in Firebird 3. A single Firebird process
opens database files in exclusive mode. Connections
served by this process share the page cache among them.
Each database has a single page cache.
• SuperClassic (or ThreadedShared) – A single Firebird
process opens database files in nonexclusive mode, so
embedded connections can access a database already in use
by SuperClassic. Each connection has a private page cache.
• Classic (or MultiProcess) – a new independent Firebird
process is started for every connection. Each process
opens the database file in a non-exclusive mode that also
allows access to embedded servers. Each connection has
a private page cache.

 The old aliases.conf file was renamed databases.conf.


It offers a much wider range of settings than the
old aliases.conf file, which only defined aliases for
databases.

32 vs. 64 bits

Firebird offers both 32 and 64-bit native binaries. Obviously, a


32-bit operating system cannot run 64-bit Firebird. However, if the
operating system is 64 bits, it can run either the 32 or the 64-bit
version of Firebird.

Basic but essential concepts!


26

 The 64-bit Firebird may consume 30% more


memory than the 32-bit version. When using the
64-bit version, make sure that the amount of
RAM on the server is generous.

The main difference between 32 x 64 bit versions of Firebird is


the amount of memory that Firebird can access and, consequently,
how far it can scale. 32 bit Firebird cannot address more than 2 GB
of memory. On a server that has +4 GB of RAM, running a 32-bit
version of Firebird cannot address all available memory, which hurts
performance especially in SuperServer and SuperClassic architectures.
When choosing between 32 and 64-bit versions of Firebird
consider your use of UDFs. Most legacy UDFs available on Internet
were compiled in 32 bits and cannot be used with 64 bit Firebird.
Recompiling the UDFs with a 64-bit compiler solves that problem,
but the source code of some UDFs is not available, and there is no
64 bits compiler available for some languages or operating systems.
The default Firebird UDFs (fbudf and ib_udf) are compiled in both
"bitnesses" and installed appropriately. UDFs created by third
parties cause most of the problems!

 The FreeAdhocLib package can replace many UDF


packages available on the Internet. This library of
UDFs offers a series of functions fully compatible
with legacy UDF packages and is available for
both 32 and 64-bit architectures. FreeAdhocLib can
be found at freeadhocudf.org.

 Pay special attention in the version of the Firebird


client library (fbclient.dll) loaded by the client
application. For 32-bit applications, the fbclient.dll
must be 32 bits. For 64-bit applications, the client
library must be 64-bits. The 64 bit Firebird 3
installer includes a copy of the 32-bit version of
the client library, in the WOW64 folder of the
Firebird installation directory.

Basic but essential concepts!


27

Installing Firebird 3
Learn how to install Firebird 3 on Linux and Windows.

Installing Firebird 3
28

Installing Firebird 3 on Linux

The combination of Firebird + Linux is very powerful and


stable! Here's how to install Firebird 3 on Linux CentOS 7 (64-bit),
using the .tar.gz file. On some Linux distributions, Firebird 3 is
available in the repositories.

 I’m far from a Linux expert. This chapter is a


basic step-by-step guide to one Linux installation.
Different Linux distributions have different rules
for installing layered products, so what you see in
this guide may not match your experience
installing on your Linux system.

The first step is to visit the Firebird site – www.firebirdsql.org –


and click in the Downloads option in the top menu, followed by the
Firebird 3 link at the left side menu.
The Downloads page lists numerous files for different operating
systems, architectures (32, 64-bit), zip kits, installers, and more. The
PDB files help the core-developers debugging problem at users'
sites.

 Firebird 3 requires these minimum versions of


glibc and Linux kernel:
• kernel - 2.6.39
• glibc - 2.12

Installing Firebird 3
29

 Before continuing, read the “Basic concepts”


chapter. The information there helps you make
good decisions during the installation.

In this how-to, I used the Firebird-3.0.0.32366-


ReleaseCandidate2.amd64.tar.gz file, the most recent version available
when this book was written.
After downloading the file, extract its contents. Open a shell
(terminal) on Linux, then go to the directory where the download file
was saved and run the following command:
tar -xvf Firebird-3.0.0.32366-ReleaseCandidate2.amd64.tar.gz

That command creates a new directory with the same name as


the file "Firebird-3.0.0.32366-ReleaseCandidate2.amd64", containing the
extracted files, including the installation script (install.sh).
Go to the newly created directory and run install.sh:
cd Firebird-3.0.0.32366-ReleaseCandidate2.amd64/
sudo ./install.sh

 You must log in as root or use sudo to obtain root


privileges. Otherwise, you see the error “You need
to be 'root' user to do this change”.

Hit ENTER to start the installation.


At this point, you may find some “surprises”. In this example,
the installation failed because the libtommath library was missing and
the script depends on it. If the library is present on your system, the
installation process continues. However, in this example, we need to
solve the dependency problem before continuing.

Installing Firebird 3
30

Let’s install the libtommath library:


wget http://dl.fedoraproject.org/pub/epel/6/x86_64/libtommath-
0.42.0-3.el6.x86_64.rpm
sudo rpm -Uvh libtommath-0.42.0-3.el6.x86_64.rpm

 You can install the library from the yum repository


using:
sudo yum update yum install libtommath

When you have resolved the library dependency, restart the


installation, running the install.sh one more time:
sudo ./install.sh

Firebird 3.0.0.32366-ReleaseCandidate2.amd64 Installation

Press Enter to start installation or ^C to abort


Extracting install data
Updated /etc/services
Please enter new password for SYSDBA user: masterkey
Install completed
[osboxes@osboxes Firebird-3.0.0.32366-ReleaseCandidate2.amd64]$

Installing Firebird 3
31

The installer asks you for the SYSDBA password. This example
used the password masterkey, which was the default password for the
SYSDBA account for decades. Never use that password in
production servers. It is widely known and therefore very insecure.
If everything went fine, the installation finishes with “Install
completed” message.

 You can run the installation script using the -silent


parameter, for a non-interactive installation which
does not ask any questions. The script generates a
random password for the SYSDBA and stores it
in the SYSDBA.password file.

 Many users prefer to install Firebird from


repositories using, for example, apt-get (in distros
based on Debian) or yum (in distros based on
RedHat). Using repositories can be easier than
installing a downloaded kit, since missing
dependencies are resolved automatically.
Repositories also simplify updating installed
software. However, the new Firebird release may
not be available in repositories immediately.

To verify that Firebird installed correctly, connect to the example


database (employee.fdb), present in all Firebird installations.
Go to the Firebird’s installation directory:
cd /opt/firebird

 Depending on the Linux distro used, the


installation base directory can be different from
/opt.

Try connecting to the employee database using isql:

Installing Firebird 3
32

./bin/isql employee
Statement failed, SQLSTATE = 08006
Can not access lock files directory /tmp/firebird/
Use CONNECT or CREATE DATABASE to specify a database
SQL> quit;

If you see that error, your login does not have sufficient rights to
create an embedded connection to Firebird. Because there was no
host name in the command line, Firebird attempted to create an
embedded connection, rather than a network connection over tcp/ip.
Remember an embedded connection does not validate the user and
password.

 As installed, Firebird has an alias called employee,


pointing to the example database (employee.fdb)

There are three possible ways to connect to the example


database:

1. Execute isql as root (i.e. sudo ./bin/isql employee)


2. Add the current user to the Firebird group on Linux
(sudo usermod -a myuser -G firebird). If you add the
current user to the group, you must login again for the
change to take effect.
3. Connect to employee via tcp/ip (localhost), instead of using
an embedded connection.

Using the third option:


./bin/isql -user SYSDBA -pas masterkey localhost:employee
Database: localhost:employee, User: SYSDBA
SQL> show database;
Database: localhost:employee
Owner: SYSDBA
PAGE_SIZE 8192
Number of DB pages allocated = 307
Sweep interval = 20000
Forced Writes are ON
Transaction - oldest = 165
Transaction - oldest active = 166
Transaction - oldest snapshot = 166
Transaction - Next = 171
ODS = 12.0
Default Character set: NONE

Installing Firebird 3
33

Since we were able to connect to the employee database, we have


demonstrated that Firebird is operational.
By default, the installer sets Firebird to run in SuperServer mode.
The server mode can be changed by setting the ServerMode parameter
in firebird.conf to Super, Classic, or SuperClassic. However, on Linux,
you should change server modes using the
changeServerMode.sh script, available in the Firebird’s bin
directory.
What is the difference?
Alex Peshkov says that when Classic mode is selected by
modifying the ServerMode parameter in firebird.conf, Firebird itself acts
as the connection listener. This is a new way to run Classic, and it has
not been widely tested in production environments to confirm its
stability. In older Firebird versions, Classic depended on the
inetd/xinetd (en.wikipedia.org/wiki/Xinetd) to receive connection
requests and forward them to Firebird.
The changeServerMode.sh script sets Firebird to Classic mode, and
makes all the changes necessary to use xinetd or systemd. One
advantage of xinetd is the ability to restrict the number of active
connections or even to reject new connections.
sudo ./changeServerMode.sh
Firebird server may run in 2 different modes - super and classic.
Super server provides better performance, classic - better
availability.

Which option would you like to choose: (super|classic) [super] classic


Stopping currently running engine...
Starting firebird in classic server mode...
Done.

 If you change the Firebird server mode using the


changeServerMode.sh script, all future changes must
also use the script. Otherwise, you may end up
with an inconsistent installation. On Linux, you
should always change server modes through
the script. Do not change the ServerMode
parameter in firebird.conf directly.

Installing Firebird 3
34

 The installation on Ubuntu 14.04 LTS and 15.03


64-bit is similar to the CentOS, installation,
including the problem of the missing libtommath
library. Since Ubuntu doesn’t have rpm or yum,
install the missing library with the following
commands:
sudo apt-get update
sudo apt-get install libtommath

To uninstall Firebird, use the FirebirdUninstall.sh script, located in


the Firebird bin folder.

 If you have an older Firebird version installed and


want to run Firebird 3 concurrently, follow this
procedure:

1. Download the Firebird 3 .tar.gz file


(Firebird-3.0.0.32366-
ReleaseCandidate2.amd64.tar.gz) from the
official site and extract the contents.
2. Extract the contents of the buildroot.tar.gz
file, extracted in the previous step.
3. Copy the content of the opt/firebird folder
which was extracted in the previous step
to another directory, e.g. /opt/firebird3
4. Edit firebird.conf in opt/firebird3 changing
parameters that conflict with existing
Firebird installations, e.g.
RemoteServiceName, RemoteServicePort, etc.
5. Create the SYSDBA user.
6. Start Firebird 3.

Installing Firebird 3
35

Installing Firebird on Windows®

Installing Firebird on Windows still is very simple! The installer


keeps the “Next-Next-Finish” tradition and, in a few minutes, you
have a full working Firebird installation, ready for use!

 Before continuing, read the “Basic concepts”


chapter. The information there helps you make
good decisions during the installation.

The first step is to visit the Firebird site – www.firebirdsql.org –


and click in the Downloads option in the top menu, followed by the
Firebird 3 link at the left side menu.
The Downloads page lists numerous files for different operating
systems, architectures (32, 64-bit), zip kits, installers, and more. The
PDB files help the core-developers debugging problem at users'
sites.
For this tutorial, I installed the 32-bit version of Firebird 3 RC2,
the most recent version available when this book was written.
Run the installer with administrator rights. The first screen lets
you choose the Language used during the installation process. Let’s
choose English:

Installing Firebird 3
36

The next screen displays the Firebird license. Accept it and click
Next to continue.

The next screen presents important pre-installation information.


Click Next to continue.

Installing Firebird 3
37

Next, you choose where to install Firebird 3. The default on


Windows is (Program Files folder)\Firebird\Firebird_3_0. In this how-
to, I install it in C:\Firebird3

Installing Firebird 3
38

In the following step, you choose among the four different types
of installation:

• Full installation of Server and development tools


– Installs Firebird server, client, and the command
line utilities, gbak, gfix, isql, etc.
• Installation of the Client tools for Developers
and database administrators – Install the Firebird
client library for remote access to a Firebird server
and the command line tools, gbak, gfix, isql, etc.
• Minimum client install – no server, no tools –
Install only the files necessary for remote access to a
Firebird server: the fbclient.dll (Firebird’s client
library).
• Custom installation – allows a choice of modules
to install: Server components, Developer and admin tools
components, or Client components (the last is required).

Choose “Full installation”.


The installer asks which folder in the Windows’s Start Menu
should display the Firebird icons. You can choose not to create any
icons.

Installing Firebird 3
39

The next screen deserves special attention. On it, you choose


several important options:

Installing Firebird 3
40

Server architecture

You choose among Classic, SuperClassic or SuperServer, which is


the default. When choosing SuperClassic or SuperServer, you may opt
to use the Firebird Guardian process to start, restart, and monitor the
Firebird server. Using the Firebird Guardian is not recommended
when Firebird is installed as a service.

Service or Application?

Unless you have a very specific reason not to, you should run
Firebird as a Service, which is the default mode. Running Firebird
as a service avoids the need to log in on Windows and start Firebird,
with the chance of starting it from the wrong account. Running
Firebird as a Service eliminates the need for the Firebird Guardian.
Running Firebird as an Application on development systems can
make it easier to start and stop Firebird frequently. Having the
Firebird icon on the Windows taskbar also makes development
easier.

Start automatically

In a production server, Firebird should start automatically when


Windows starts. If this option is not chosen, someone must start
Firebird manually after the operating system boots.

Client library (fbclient.dll)

The installer allows you to copy the fbclient.dll file to the


Window’s system directory either system32 or syswow64, depending
on the “bitness” of the operating system. Copy the client library to
the system folder only if you use legacy applications that cannot find
the client library in another folder.

Installing Firebird 3
41

gds32.dll

Legacy applications created to work with InterBase or Firebird


1.0 expect the client library name to be gds32.dll. If you use ancient
legacy applications, mark this option so the installer creates the
backward compatible library.

Authorization for legacy Firebird clients

This option configures the AuthServer, AuthClient, UserManager


and WireCrypt parameters to allow pre-Firebird 3 fbclients to connect
to Firebird 3 servers. Note that choosing this option makes
Firebird 3 less secure.

Proceeding with the installation, the next step allows you to


define a name and password for the administrator user. If you leave
these fields blank, the installer creates an administrator user
SYSDBA, and set the password to masterkey:

Installing Firebird 3
42

 Never use masterkey as a password on


production servers or servers open to the
internet. masterkey was the default SYSDBA
password for decades and is very insecure.

The next screen displays the options chosen during the


installation. This is a last chance to review the options and change
your mind:

If everything is OK, click Install. The installer copies files and


configures the server.
The next screen shows post-installation information for this
Firebird version:

Installing Firebird 3
43

Finally, you can choose to start Firebird server and open the
“What’s next” page, which requires an Internet connection.

Checking whether Firebird is running

You can check whether Firebird is running in several ways.


If you opted to install Firebird as an application, its icon appears
in the Windows taskbar when Firebird is running.
If Firebird is running as a service, there is no icon in the
Windows taskbar. In this case, just run the Window’s Services
application, and check the Firebird’s service status, as shown in the
following image.

Installing Firebird 3
44

Firebird’s instsvc utility, with the “q” option can also verify the
service status:
C:\Firebird3>instsvc q
Firebird Server - DefaultInstance IS installed.
Status : running
Path : C:\Firebird3\firebird.exe -s DefaultInstance
Startup : automatic
Run as : LocalSystem

Another way to check whether the server is running is to connect


to a database, like the sample database employee.fdb, which comes
with every Firebird installation.
Open a command prompt and try to connect to the employee.fdb using
the isql utility:
C:\Firebird3>isql -user SYSDBA -pas masterkey localhost:employee

Installing Firebird 3
45

That command line connects to the employee.fdb database via tcp/ip


(localhost), using the alias “employee”, which is defined in the
databases.conf file at installation time.
If Firebird server is not running, you see this error message:
C:\Firebird3>isql -user SYSDBA -pas masterkey localhost:employee
Statement failed, SQLSTATE = 08006
Unable to complete network request to host "localhost".
-Failed to establish a connection.
Use CONNECT or CREATE DATABASE to specify a database

Otherwise, you see isql prompt, indicating that the Firebird server
is operational. The show tables command lists the tables of the
database, proving that the connection is live:
Database: localhost:employee, User: SYSDBA
SQL> show tables;
COUNTRY CUSTOMER
DEPARTMENT EMPLOYEE
EMPLOYEE_PROJECT JOB
PROJECT PROJ_DEPT_BUDGET
SALARY_HISTORY SALES

To quit isql and return to the Window’s command prompt, type:


exit; <enter>

 By default, Firebird uses the TCP/IP port


3050. If the Windows firewall or a third party
Firewall is active, you must open port 3050;
otherwise, connection attempts coming from
network terminals fail.
The Windows Firewall can be configured through
the command prompt, running the following
commands as Administrator:

In Windows Vista (or above):


netsh advfirewall firewall add rule name="Firebird"
dir=in action=allow protocol=TCP localport=3050

In Windows XP:
netsh firewall add portopening protocol=TCP
port=3050 name="Firebird" mode=ENABLE scope=SUBNET

Installing Firebird 3
46

Installing Firebird using the “Zip Kit”

The Zip Kit is simply the directory structure and files necessary to
run the Firebird server, compressed into a single zip file. You can
find it on the Downloads page of the Firebird official site,
firebirdsql.org.
In some cases, installing Firebird using the installer is too
restrictive. The installer cannot allow you to run more than one
Firebird version on a single computer.
Unpacking the Zip Kit is only the start of the installation process.
The Firebird kit includes utilities that help you complete the
configuration: instsvc, instreg and instclient.

INSTSVC

This utility installs, configures and manipulates the Firebird


service on versions of Windows that support running Firebird as a
service.
The command that installs the Firebird service is instsvc
install. It creates the Firebird service, configures it to start
automatically and use the standard system identity called LocalSystem.
Instsvc can customize the service using these commands and
switches:
instsvc i[nstall]
[ -a[uto]* | -d[emand] ]
[ -g[uardian] ]
[ -l[ogin] username [password] ]
[ -n[ame] instance ]
[ -i[nteractive] ]
sta[rt] [ -b[oostpriority] ]
[ -n[ame] instance ]
sto[p] [ -n[ame] instance ]
q[uery]
r[emove] [ -n[ame] instance ]

You can read the full description of the command arguments in


the README.instsvc.txt file in Firebird’s doc folder. Items marked
with an * indicate the default action.

Installing Firebird 3
47

 Associating the service with a different user

The default Firebird service installation leaves the


service associated with the user "LocalSystem". This
may open a window for hackers trying to gain
access to the computer by attacking the Firebird
service. For better security, associate the service
with a different user, one with restricted privileges.
The instsvc can configure the service to associate it
with an existing Windows user. This user must
have read/write access to all database files and to
the file firebird.log. For security reasons, the user
should not have the right to change firebird.conf or
the Firebird executables.
Give instsvc the user name/password to be used
with the Firebird service, by including the option -
login followed by the user name and password
separated by a space, for example:
instsvc i -g -login myuser mysecret

 In installations using Firebird Guardian, the Firebird


service must be configured to start manually, since
the Guardian restarts it after an operating system
reboot or server crash.

Instsvc can start/stop the Firebird service:


instsvc start
Service "Firebird Server - DefaultInstance" successfully started.

instsvc stop
Service "Firebird Server - DefaultInstance" successfully stopped.

INSTREG
The instreg.exe utility modifies the Windows registry, creating the
Firebird default key (HKLM\SOFTWARE\Firebird Project\Firebird
Server\Instances), which points to the Firebird installation directory.
The server process does not require this key, but a number of client
applications, including Firebird's own utilities, use this key to find
the Firebird directories.

Installing Firebird 3
48

The base Firebird installation directory configured in the registry


key is the folder where the instreg.exe is located, so instreg.exe must
be stored and run in the Firebird folder.
Instreg parameters are:
instreg i[nstall]
r[emove]

Where:

i[nstall]: create the key in the Windows registry.


r[emove]: delete the key from the registry.

INSTCLIENT

Instclient.exe places a copy of the Firebird client library (fbclient.dll)


or the backward compatibility library (gds32.dll) in the Windows
system folder.
instclient i[nstall] [ -f[orce] ] library
q[uery] library
r[emove] library
where library is: f[bclient] | g[ds32]

I[nstall] copies the chosen client library. The -f[orce] option


causes the utility to copy the library, even there is a newer library
already installed. Use the force option with caution, since existing
installations may depend on the newer client library.
R[emove] deletes the previously deployed library.
Q[uery] displays the version information of the library.

Installing Firebird 3
49

Migrating Existing
Databases to
Firebird 3

Migrating Existing Databases to Firebird 3


50

Why Migration?

Firebird 3 can only access databases that it creates. Previous


versions of Firebird were backward compatible. They could read
versions of the database created by older versions of Firebird and
even some created by Interbase. New features in Firebird often
require changes to the storage format of the database. Maintaining
the ability to access many different old versions of the database
structure complicates the code of the Firebird server. That code
underwent major changes in Firebird 3 to allow queries to run in
parallel on a multi-core machine in SuperServer mode and, as a
result, backward compatibility was dropped.
In order to use existing databases under Firebird 3, you must
convert them to the new storage format. Generally, that conversion
can be done simply by using gbak to back up the database running
your current version of Firebird and restore it running Firebird 3.
However, several issues may complicate the migration, especially
from older versions of Firebird. If you have not made a practice of
backing up and restoring your databases with gbak, you may find
that errors have crept into the physical or logical structure of the
database that make the migration more difficult.
Before attempting to migrate your databases, you should first
verify that you can back them up and restore them under your
current version of Firebird. Backing up the database should be
simple, but can reveal hidden problems in the physical structure of
the database that can be fixed with the gfix utility. Restoring the
database can uncover problems in the logical consistency, such as
nulls in non-null columns, duplicates in unique indexes, and
violations of referential constraints. These problems are more likely
to occur if you are running older and less reliable versions of
Firebird, or used to manipulate the system tables directly. The gbak
error log will identify the errors. You can fix them with isql or any
data-browsing tool.
As Firebird has expanded and improved its SQL implementation,
some parts of the language have changed. Firebird 3 has reserved
words that were not reserved in earlier versions. If your database
uses those words as the names of tables, fields, views, or in triggers
or stored procedures, you must change your data definitions before

Migrating Existing Databases to Firebird 3


51

you can migrate to Firebird 3. Firebird 3 may also be more rigorous


in identifying non-standard SQL than earlier versions. To find SQL
language problems in your database, you can extract the database
definition under your current version of Firebird and use that script
to create a new database using Firebird 3.
Some older versions of Firebird did not handle non-ASCII
characters in system tables consistently and allowed non-standard
characters to appear in metadata. The gbak error log for Firebird 2.1,
2.5, or 3 identify those errors and special switches on gbak can
correct them in most cases. If you are not running Firebird 2.1 or
later you can delay correcting character set mangling until the actual
migration to Firebird 3.
Once you have established that your database is physically and
logically correct and that your metadata definitions meet the SQL
standards of Firebird 3, you are ready to start migrating the
database.
The remainder of this chapter describes in more detail the history
of Firebird's On Disk Structure, techniques for identifying potential
backup/restore problems and SQL language problems, and the best
method for the final migration of your database. You should follow
all of the steps for each database you need to migrate to Firebird 3
to avoid having your database unavailable for longer than necessary.

ODS (On Disk Structure)

Every Firebird database has a well-defined structure. The ODS


(On Disk S tructure) version number stored on the database header
page declares which version of the ODS is used in that database.
The table below lists the Firebird versions and the associated
ODS version.

Firebird ODS
1.0 10.0
1.5 10.1
2.0 11.0
2.1 11.1
2.5 11.2
3.0 12.0

Migrating Existing Databases to Firebird 3


52

The ODS is represented by a number in the format of MM.m,


where MM is the Major Version, and m is the Minor Version, for
example: 10.1 (M = 10, m = 1). Major Versions represent large
changes to the ODS which introduce major new capabilities.

 To determine which ODS a database implements,


use the gstat utility with the -h switch, and look for
the value in "ODS Version" line

Prior to Firebird 3, all versions of Firebird could access databases


created with an older ODS. You can use the Firebird 2.5 to access a
database created in Firebird 1, without any problem. However,
Firebird 3 can access only ODS 12 databases.

 Firebird 3 cannot access databases created with


earlier Firebird versions. Only databases with
ODS 12 are recognized.

Therefore, when migrating to Firebird 3, you must use the gbak


utility to backup the database using your currently installed version of
Firebird. Then, when Firebird 3 is running, you should restore the
backup, creating a new database with ODS = 12.
In a future update, the Firebird development team hopes to add
a provider that can access databases created with older ODS
versions. However, there is no schedule for delivery of that provider
and it may never appear. For now, you must migrate your databases
to ODS 12 to use Firebird 3.

Test the database integrity with gbak

Before you attempt to migrate a database from an earlier version


of Firebird, use gbak to backup and restore the database. You can
make the backup with the database active, though you should
choose a relatively slow time to avoid annoying other users (specially
with huge databases).
With your current version of Firebird installed, perform the
following steps, where user is the owner of the database, password is
the password for that user in Firebird, database is the full path to the

Migrating Existing Databases to Firebird 3


53

database file on the server, and backupfile is the full path to the
backup file to be generated on the server:
gbak -user user -pas password -b -v -g -se service_mgr database
backupfile

If the gbak backup fails indicating a possible corruption in the


database, take your database off-line and make a physical copy of
the file for safety. Run the gfix utility to validate the database:
gfix -v -full -user user -pas password database

If gfix reports transactions in limbo, run it again, this time using the
-mend switch.
Once you have a good backup, restore it:
gbak -user user -pas password -c -v -se service_mgr backupfile
database

 Be sure to restore your database under a different


name. Overwriting a database is a recipe for
disaster.

Check the output during restore process! If gbak reports duplicated


primary or unique keys, or violations of foreign key constraints when
trying to create the indexes used by the integrity constraints, it prints an
error. The restore continues to run. At the end, affected indexes are
inactive, and the database is shutdown in single user mode - offline. You
must correct the data problems manually (removing the duplicates,
broken records or recreating their masters), then activate the foreign
keys (alter index <name> active), and put the database file back
online using gfix -online normal.

 Correcting problems before you start migrating to


Firebird 3 reduces the time that the database must
be offline.

If you are running Firebird 2.1 or later, you should check for
malformed strings while you are validating that you can backup and
restore the database. The next section describes that check.

Migrating Existing Databases to Firebird 3


54

Problems with character encoding

If you currently run Firebird 2.1 or later, you should perform this
check before starting to migrate to Firebird 3. If you are using an
older version of Firebird that mishandles conversions from other
character sets to UNICODE_FSS, you can wait until the migration
step to perform this check and correct errors.
Firebird uses the character set UNICODE_FSS to store the data
in the system tables (RDB$...), strings, and source code of objects like
procedures, triggers, etc.
Before Firebird 2.1, bugs in Firebird caused it not to convert
multi-byte character sets sent by the client to UNICODE_FSS. As a
result, that data was stored in an invalid format in columns declared
as unicode_fss, whether in user data or metadata.
The Firebird 2.1 release refused to accept those malformed strings.
If a database incorrectly coded metadata strings, restoring a backup
reports an error about malformed strings, either for metadata problems
or for data, for example:
gbak: ERROR:Malformed string
gbak:Invalid metadata detected. Use -FIX_FSS_METADATA option.

To solve the problem, two command line switches were added to


gbak in Firebird 2.5, that can correct malformed strings during a restore:
-fix_fss_metadata and -fix_fss_data. For Firebird 2.1, you need to
use the scripts located in misc/upgrade/metadata.
Check whether the error message refers metadata, data, or both.
Repeat the restore, using -fix_fss_metadata , or -fix_fss_data or
both. Each switch must be followed by the name of the character
set used to insert the problematic metadata or data e.g. win1252, utf8,
etc. To use the switches to correct malformed strings automatically, you
must know what character set was used to insert the original data.

 Use the option -fix_fss_data only if columns in the


database are defined with the character set
unicode_fss.

These switches cause the metadata or data to be converted from


the named character set and stored again using unicode_fss.

Migrating Existing Databases to Firebird 3


55

You cannot use the -fix… switches if a database has encoding


problems with objects inserted in different character sets. You can
correct those errors manually using scripts available in the
misc/upgrade/metadata folder of Firebird 2.1/2.5.

 Do not use the -fix_fss_metadata or -fix_fss_data


switches more than once on a database. That can
cause corruption!

Remember: use these switches only if the first try to run the
restore had failed due to “malformed string” errors! If you use them
when they are not needed, you can corrupt the database.

Validating the metadata

Before migrating a database to Firebird 3, you should check for


compatibility with the new version of the RDBMS.
For example, Firebird 3 has added reserved words. The gbak
restore recognizes table and column names that are reserved words
and changes them to quoted identifiers. However, triggers, stored
procedures, validity constraints, and even view definitions may contain
Firebird 3 reserved words that are not caught and corrected by gbak.
Any of these issues can invalidate the database’s metadata.
Unfortunately, you discover the problem only when you try to
recompile the code of procedures/triggers that reference names,
variables, or aliases that are now reserved words.
An easy way to validate the metadata of an existing database is to
extract it to a script and run the script to generate a new database on
a Firebird 3 server. Any inconsistency in the metadata causes an
error during the script execution. You can fix it either by changing
to a word that is not reserved or by double quoting the reserved
word. Quoted identifiers are a feature of SQL Dialect 3.
After you correct the metadata and test it with Firebird 3, you
have created a different script that creates a slightly different
database. You can migrate the database in one of two ways:

1. Create a script that corrects the database and run that


script against the restored database.

Migrating Existing Databases to Firebird 3


56

2. Create a new database in Firebird 3 with the validated


script and use a data pump utility to copy the data from
the old database to the new one.

Depending on the complexity of your metadata, the nature of the


problems, and the corrections you made, pumping data may be the
easiest way to migrate your database. IBDataPump is one tool for
this job: clevercomponents.com/downloads/datapump/index.asp.

 Take the opportunity to check that the database


metadata is fully compatible with version 3 of
Firebird and avoid future surprises!

Below we have an example of problematic metadata:


CREATE TABLE ATABLE (
CORR INTEGER,
CAMPO_A VARCHAR(100)
);
CREATE TABLE OFFSET (
CAMPO1 INTEGER,
CAMPO2 VARCHAR(100)
);
SET TERM ^ ;
CREATE PROCEDURE TEST
AS
declare variable OVER integer;
begin
select A.CORR
from ATABLE A
into :OVER;
end^
SET TERM ; ^

The highlighted words are reserved in Firebird 3 but not in


Firebird 2.1. When you run a backup on the old server and a restore
on the new server, gbak does not report errors. Instead, gbak
changes object names that conflict with reserved words to quoted
identifiers.
Below we can see the metadata of the same database restored in
Firebird 3 and extracted by isql:

Migrating Existing Databases to Firebird 3


57

SET SQL DIALECT 3;


/* CREATE DATABASE 'd:\TESTEMETADATA.FB3' PAGE_SIZE 8192
DEFAULT CHARACTER SET WIN1252; */
COMMIT WORK;
SET AUTODDL OFF;
SET TERM ^ ;
/* Stored procedures headers */
CREATE OR ALTER PROCEDURE TEST AS
BEGIN EXIT; END ^
SET TERM ; ^
COMMIT WORK;
SET AUTODDL ON;
/* Table: ATABLE, Owner: SYSDBA */
CREATE TABLE ATABLE ("CORR" INTEGER,
CAMPO_A VARCHAR(100));
/* Table: OFFSET, Owner: SYSDBA */
CREATE TABLE "OFFSET" (CAMPO1 INTEGER,
CAMPO2 VARCHAR(100));
COMMIT WORK;
SET AUTODDL OFF;
SET TERM ^ ;
/* Stored procedures bodies */
ALTER PROCEDURE TEST AS
declare variable OVER integer;
begin
select A.CORR
from ATABLE A
into :OVER;
end ^
SET TERM ; ^
COMMIT WORK;
SET AUTODDL ON;

Note that the reserved words CORR and OFFSET appear in the
script enclosed in double quotes ("") making them quoted
identifiers, but only when they are the names of tables, fields,
procedures, etc. that are being defined. Gbak restore does not "auto-
correct" when recreating the source code of the TEST stored
procedure, which references the reserved words CORR and OVER!
The TEST procedure runs successfully in Firebird 3, because the
backup/restore process preserves the BLR (B inary L anguage
R epresentation) version of the procedure created on the previous

Migrating Existing Databases to Firebird 3


58

server version rather than recompiling procedures and triggers from


source.
However, when you try to create a new database in Firebird 3,
using this same script extracted by isql Firebird reports errors:
C:\firebird3>isql -i d:\script.sql
Use CONNECT or CREATE DATABASE to specify a database
Statement failed, SQLSTATE = 42000
Dynamic SQL Error
-SQL error code = -104
-Token unknown - line 2, column 18
-OVER
At line 33 in file d:\script.sql

The execution of the script failed at line 33:


declare variable OVER integer;

A similar problem occurs if a developer alters the source code of


the TEST procedure, without enclosing OVER and CORR in
double quotes:
ALTER PROCEDURE TEST AS
declare variable "OVER" integer;
begin
select A."CORR"
from ATABLE A
into :"OVER";
end ^

 Checking the metadata ensures that the database


definition is compatible with Firebird 3, but does
not guarantee that your application is also
compatible. If you have found problems with
reserved words in the metadata, you must check
the selects, updates, inserts etc. run by the application
and change any that use reserved words.

For more information, see the item "Reserved words" in the


chapter " Firebird 3 and existing applications”.

 Remember that referencing objects using double


quotes makes them case sensitive, so you must

Migrating Existing Databases to Firebird 3


59

reference the objects in exactly the way that they


were declared.

Migrating a database to Firebird 3

If you followed the previously described validation processes, the


following steps let you migrate an existing database to ODS 12
safely:
With your current version of Firebird installed, perform the
following steps, where user is the owner of the database, password is
the password for that user in Firebird, database is the full path to the
database file on the server, and backupfile is the full path to the
backup file to be generated on the server:

1. Stop Firebird.
2. Rename the original database file to avoid new
connections to modify the data.
3. Start Firebird.
4. Backup with gbak:
gbak -user user -pas password -b -v -g -se
service_mgr database backupfile

 Stop database access before starting the backup!


Gbak creates a backup that reflects the state of the
database when the backup started. Subsequent
work is not included in the backup and or the new
ODS 12 database.
Make a copy of the original database before
migrating it! If you overwrite the original database
when restoring the backup and the restore fails, you
can lose all your data!

Install and start Firebird 3. If you are keeping the same server
machine, uninstall the earlier version of Firebird before installing
Firebird 3. If you want to keep both versions of Firebird available
on the same server, the simplest solution is to disable/remove the
older service while running Firebird 3. If you want to run both
versions concurrently, you must configure them manually, setting

Migrating Existing Databases to Firebird 3


60

different TCP/IP ports and different service names for each of


them. Read the chapter in this book about the installation process
for more information.

 Before restoring the backup on Firebird 3, you


must initialize the security database and create the
needed users.

1. Restore the backup to the working directory and file


name:
gbak -user user -pas password -c -v -se service_mgr
backupfile database
2. If the restore process finishes without errors, you have a
database with ODS 12 ready to work with Firebird 3.

 The -se service_mgr parameter causes the


backup to run inside the Firebird process. That
can speed up the backup/restore time by 30%.
Using the Services API creates the backup file on
the server. The database path and the backup file path
must be valid on the server.

Migrating 24x7 servers

Migrating 24 x 7 production servers requires more effort in the


migration process. As we know, Firebird 3 cannot open databases
with ODS less than 12, forcing the user backup/restore the
database to gain the benefits of Firebird 3. . Depending on the size
of the database, this process can take hours to complete! If you do
not have a sufficient time window to carry out the process, the
situation is complicated.
The suggestion for these cases is to implement a data replication
scheme, replicating the data in real time from the "old" server to the
Firebird 3 server. When the databases have synchronized fully,
redirect applications to the new server and retire the old server.

Migrating Existing Databases to Firebird 3


61

A rolling upgrade using replication requires extra work, especially


for those who are not currently using replication. However, adding
replication as a permanent feature has the benefit of serving as a "hot
backup" solution. If a disaster happens to the production server, you
can redirect applications to the replica.
As a bonus, with the two servers active during the migration, the
replica database could be "tested" to ensure that queries used by the
applications run correctly with Firebird 3.

Tips to speed up the backup/restore process

Here are a few tips to accelerate the process of doing a backup


followed by a restore:

1. Use the -g switch during the backup, to not waste time


doing the garbage collection in the “old” database.
2. Use the -se service_mgr switch, for both the backup and the
restore. That causes the backup/restore to run in the
Firebird process eliminating communication time.
3. Use SSD (Solid State Drives), which are faster than
mechanic hard drives.
4. Do not allow any other activity that consumes disc I/O
during the backup or restore.
5. If the database has huge tables with indexes and the
server has plenty of RAM, increase the parameter
SortMemSize in firebird.conf temporarily, so Firebird can use
more memory for sorting when creating the indexes
during the restore. After the restore finishes, adjust the
parameters to a more appropriate value for regular use.
Setting the value too high causes Firebird to consume all
available RAM. Then the operating system page swaps,
degrading system performance enormously.

Migrating Existing Databases to Firebird 3


62

Users in
Firebird 3
Firebird 3 introduces big changes in users management and in the
role of the SYSDBA user. Be prepared to revisit your assumptions.

Users in
Firebird 3
63

Firebird 3 brings significant changes in the management of


database users. Up to version 2.5, Firebird users were centralized on
a per-server basis. Their names and password hashes were stored in
the file isc4.gdb, security.fdb, or security2.fdb, depending on the version
of Firebird.

 The default name for the security database in


Firebird 3 is security3.fdb

The centralized user model has pros and cons:

Pros: The user management can be simpler, especially if the


same users access multiple databases on a server. In this case, you
just define the users once and assign them access rights to the DB
objects in each database. It is not necessary to create the users in
each database that they will access, simplifying system-level
management.
Cons: Any existing user can log into any database in the server
by knowing the alias or the file path. Grants control access to existing
database objects (tables, views, procedures, etc.), but there is no way
to prevent any user from creating new objects in any database.
A central security database provides easier management of users,
but loses the ability to control which databases a user can access,
compromising security.

 You cannot migrate the security database of


older Firebird versions to Firebird 3. All users
will need to be recreated on Firebird 3.

Local users

Firebird 3 introduces the concept of local, per database, users.


Instead of forcing the centralized user model, Firebird 3 allows you
to decide whether user information should be stored locally in the
database file or in an independent database.
To implement this, Firebird now includes a table structure similar
to the one found in security3.fdb, in every ODS 12 database.

Users in
Firebird 3
64

Firebird 3 also allows you to give the security database any name
you want. You can decide that your central user database will be
centralusers.fdb, instead of the default security3.fdb, and put it in any
directory on the server, not just the Firebird installation directory.
To support these new capabilities, Firebird added a new
parameter SecurityDatabase . By default, this parameter is defined
in the firebird.conf file as $(root)/security3.fdb, which declares that the
security database is stored in the root of the Firebird installation
and called security3.fdb. SecurityDatabase can also be set per
database in the databases.conf file, previously called aliases.conf,
allowing you to specify a different security file for every database!
Below we have an example of a databases.conf file defining three
aliases for three databases: base1, base2 and base3. Base1 is
configured so that its user information is stored in the file
c:\DBUsers\users_base1.fdb. Base2 is configured so that its user
information is stored locally, meaning that users is managed within
the database itself. For base3, only the alias is defined, so its user
information is stored in the default security database, defined in
firebird.conf.
base1 = c:\databases\base1.fdb
{
SecurityDatabase = c:\dbUsers\users_base1.fdb
}
base2 = c:\dbs\base2.fdb
{
SecurityDatabase = base2
}
base3 = c:\dbs\base3.fdb

 You can specify the security database in the


SecurityDatabase parameter using an alias or full
path to the file.

Remember that a security database has always been and


continues to be a "normal" Firebird database. To create a new
security database, simply create a new database, using, for example,
isql:
C:\firebird3> isql -user sysdba
SQL> create database ' c:\DBUsers\users_base1.fdb';

Users in
Firebird 3
65

Note the need to enter the isql using the command line switch
-user sysdba, even if no sysdba exists yet.
Now you have a security database, but it has no users at all, not
even the SYSDBA! The next step is to create the SYSDBA – or any
other desired user – in that database:
SQL> connect base1;
SQL> create user SYSDBA password 'mypassword’;
SQL> commit;
SQL> exit;

That sequence of commands connects the newly created


database and creates a user called sysdba with the password set as
mypassword. Notice that the user name, when not enclosed in double
quotes, is not case sensitive, but the password is, so pay attention!

 Firebird 3 doesn’t require the existence of a user


named SYSDBA. For databases without a
SYSDBA, the user who creates the database is its
owner and its "administrator".

Passwords

Despite common knowledge, Firebird does not store users'


passwords in the security database. Firebird stores a hash of the
password so no one can retrieve the password itself, because it is
not written anywhere. When you log in, Firebird compares the hash
of the password used in the login with the existing hash stored in the
security database.
Remember that up to Firebird 2.5, only the first eight bytes of a
password are considered in the hash calculation. If a password had
more than eight single byte characters, Firebird ignored characters
from the nine onward! Passwords using multi-byte characters had
even fewer significant characters.
Starting with Firebird 2.0, the hash algorithm changed from DES
– Data Encryption Standard – to SHA1, raising the security level but
keeping the eight-byte limit.

Users in
Firebird 3
66

 Up to Firebird 2.5, when you defined a password


as ‘masterkey’, only ‘masterke’ was considered in
the hash calculation, so you could login typing the
password as masterkey, masterke1234,
masterkeeeeeee, etc.

By default, Firebird 3 uses up to 255 bytes of a password for


hash calculation! However, if you configure FB 3 to use the legacy
authentication model (LegacyAuth), instead of the default SRP,
Firebird hashes only the first 8 bytes of the password.

 Avoid using short passwords. A five-character


password can be discovered in seconds by brute
force while a ten-character password would take
centuries. Encourage users to include upper and
lower case letters and symbols in their passwords
and to avoid common passwords like "password"
or “12345”. In Firebird 3, up to 255 characters of
the password are included in the hash. Consider
using password generators and vaults on the client
side for optimal security.

 Using ASCII characters in user names and


passwords avoids confusion if clients log in
through interfaces using different character sets.
Firebird 3 accepts user names and passwords
defined with multi-byte characters (e.g: UTF-8).
However, users must ensure that the codepage
used in the operating system is the same as the
character set used in the SQL client application
(set names). The "complication" increases on
Windows, where the console (command prompt)
may use a different code page than GUI
applications. In Brazil, for example, the
Windows command prompt uses code page 850,
and GUI applications use 1252.

Users in
Firebird 3
67

Initializing the security database

The Firebird installer allows you to create the SYSDBA user and
set its password during the installation. However, if a problem
occurs during the creation of the security database, or if you are not
using the installer, then the security database will be "empty" - no
users defined, not even the SYSDBA.
You can initialize the security database in two ways, creating, for
example, a SYSDBA user.
The first one uses the gsec utility, using these steps:

1. Make sure that the Firebird process is not running.


2. Open a command prompt and, in the Firebird 3 root
directory, type:
gsec -user SYSDBA
gsec>add SYSDBA -pw masterkey
gsec>quit

This will create an user named SYSDBA, with password


masterkey. Note that, as strange as it may seem, you must call gsec
passing SYSDBA as the user (-user sysdba), even though that user
doesn't exist yet!

 Gsec is deprecated in Firebird 3, since you can


manage users with SQL statements. Gsec and the
Service API can manage users only in the default
security database.

The second way to initialize the security database is through isql


using the new SQL user management statements. You must connect
to an existing database to manage users with isql. Fortunately, the
default Firebird installation includes a sample database (employee.fdb),
so you can connect to it. But, how do you connect to a database, if
there are no users created in Firebird yet? Simple! Connect through
an embedded connection. For embedded connections, Firebird does not
validate the login credentials. Follow these steps to initialize the
database:

Users in
Firebird 3
68

1. Make sure that Firebird 3 process is not running so you


will connect to employee.fdb in embedded mode.
2. Open a command prompt, and call isql like this:
isql -user SYSDBA employee
SQL>create user SYSDBA password 'masterkey';
SQL>commit;
SQL>quit;
3. Start Firebird.

Note that we call isql passing only employee, instead of the full path
to the employee.fdb database. Firebird 3 predefines the alias employee in
databases.conf pointing to the sample database. The installation
process configures employee.fdb to use the default security database,
so the SYSDBA user you just created is stored in the database
security3.fdb in the Firebird root directory and can access all databases
that use that security database.
Whenever a user is created with the name SYSDBA, that user
automatically gets administrator rights, so you do not need to assign
the role RDB$ADMIN to that account.

 Although we use the password masterkey in


examples in this book, do not use that password
on production servers. It is well known and
provides no security!

Managing users using SQL

Firebird 3 brings a full set of SQL statements for managing users.


Further, SQL is the only way to manage users defined in security
databases other than the default security3.fdb.
Below is the set of SQL statements specific to manage users:
CREATE USER name {PASSWORD ‘apassword’} [ options ]
[ TAGS ( tag [, tag [, tag ...]] ) ] [USING PLUGIN name]
ALTER USER name SET [PASSWORD ‘apassword’] [ options ]
[ TAGS ( tag [, tag [, tag ...]] ) ] [USING PLUGIN name]
ALTER CURRENT USER SET [PASSWORD ‘apassword’] [ options ]
[ TAGS ( tag [, tag [, tag ...]] ) ] [USING PLUGIN name]

Users in
Firebird 3
69
CREATE OR ALTER USER name SET [PASSWORD ‘apassword’] [options]
[ TAGS ( tag [, tag [, tag ...]] ) ] [USING PLUGIN name]
DROP USER name [USING PLUGIN name];

Where OPTIONS can be:


FIRSTNAME 'string value'
MIDDLENAME 'string value'
LASTNAME 'string value'
ACTIVE
INACTIVE

The optional USING PLUGIN clause allows you to specify the


user management plugin to use in the action of the statement.
The UserManager parameter can be configured in the firebird.conf
file and at database level in databases.conf. It specifies the default
plugin used to manage users in the security database. If more than
one plugin is listed, the first on the list is the default.

 When managing users via SQL, you must log into


an existing database. The security database you are
managing is the one for the database you are
using. It may be the system-wide security database
defined in firebird.conf or a different database if the
current database uses a security database specified
in databases.conf.

The default user management plugin in Firebird 3 is SRP.

Creating users

Create a new user with either the create user or create or alter user
statement.
When creating a user, you must include at least a password for
that user. Other attributes, such as firstname, active, etc. are optional.
Firebird3 allows you to enable or disable users and define tags by
assigning a name and a value to them. A disabled user continues to
exist in the security database, but cannot login. During the creation
of the user, you can choose to create it as enabled or disabled using
the active or inactive clauses.

Users in
Firebird 3
70

 Only the SYSDBA or users that have the


rdb$admin role in the security database and are
logged in with that role can create new users.

Below we have some examples of creating users. The statements


can be run with any SQL client, including the Firebird query tool
isql. Everything that comes after a "--" is a comment:
create user Albert password 'e=mc2'; -- creates an user named
Albert with password e=mc2.

create user Albert password 'e=mc2' firstname 'Albert' lastname


'Einstein' inactive; -- creates a user named Albert with
password e=mc2, and sets the first name as Albert and the last
name as Einstein, leaving the user unable to log in (inactive).

create user Albert password 'e=mc2' inactive tags (degree


='genius', specialty = 'math');

This command creates the user Albert and specifies two tags. The
first tag is degree and set to 'genius'. The second tag is specialty and set
to 'math'. Tags are associated with the user (Albert) and can be
queried, modified, or removed when necessary.

 The values assigned to tags are of type varchar, and


cannot exceed 255 bytes.

 In previous versions, user names were not case


sensitive. They were stored in uppercase. When
you log in, Firebird compared the defined user
names with the user name provided by converting
it to uppercase.
In Firebird 3, users created by GSEC are stored
in uppercase, but if you use the SQL CREATE
USER and enclose the user name in double
quotes, it will be stored exactly as it was
typed. If you created a user containing lowercase

Users in
Firebird 3
71

characters, he will not be able to login using the


Window’s isql and the -user command line switch.
The only option will be to use the CONNECT
statement specifying the user name enclosed in
double quotes.
Passwords are case sensitive and must be
provided in the case and character set defined.

Modifying users

Only the SYSDBA or a user assigned and logged in with the role
rdb$admin for this database can alter other users. "Normal" users can
change only their own password, first name, middle name, last name, and
tags.
Examples of altering users:
alter user albert active; -- enable user albert.

alter user albert set tags (specialty = 'physics'); -- set the


tag named “specialty” assigning the value “physics” for the
user albert

alter user albert set tags (drop specialty); -- Removes the tag
"specialty" for the user albert.

When users who are not an administrator want to change their


own information, they can use a simplified form of the alter
statement, like this:
alter current user set password 'mastermind'; -- modify the
password of the current user to “mastermind”.

Deleting users

To delete existing users, use the statement DROP USER. Only


the SYSDBA or a user assigned and logged in with the role rdb$admin
for this database can delete users. The following statement deletes
the user called albert:
drop user albert;

Users in
Firebird 3
72

 User management SQL statements are


transactional. Don't forget to commit after issuing
any user create, alter or drop statements to
confirm the operations. If you use the isql set ddl
command, your statements are auto-committed.

Sec$users and sec$user_attributes virtual tables

Firebird 3 includes two new "virtual" tables, allowing you to list


existing users and their attributes. Below we can see the structure of
the table sec$users:
SQL> show table sec$users;
SEC$USER_NAME (RDB$USER) CHAR(31) CHARACTER SET UNICODE_FSS
Nullable
SEC$FIRST_NAME (SEC$NAME_PART) VARCHAR(32) CHARACTER SET
UNICODE_FSS Nullable
SEC$MIDDLE_NAME (SEC$NAME_PART) VARCHAR(32) CHARACTER SET
UNICODE_FSS Nullable
SEC$LAST_NAME (SEC$NAME_PART) VARCHAR(32) CHARACTER SET
UNICODE_FSS Nullable
SEC$ACTIVE (RDB$BOOLEAN) BOOLEAN Nullable
SEC$ADMIN (RDB$BOOLEAN) BOOLEAN Nullable
SEC$DESCRIPTION (RDB$DESCRIPTION) BLOB segment 80,
subtype TEXT CHARACTER SET
UNICODE_FSS Nullable
SEC$PLUGIN (RDB$PLUGIN) CHAR(31) CHARACTER SET UNICODE_FSS
Nullable

The fields names are self-explanatory, but two of them deserve


special attention:
SEC$ADMIN – when true, indicates that the user has the role
rdb$admin assigned in the security database, allowing him to manage
other users;
SEC$PLUGIN – Specifies the user management plugin associated
with that user;

 Firebird 3 allows the usage of several user manager


plugins (i.e.: SRP, LegacyUserManager, etc.). Each
plugin has its own users. As a result, the sec$admin
table can return "repeated" users. For example,
you can have two SYSDBAs, one managed by
Users in
Firebird 3
73

SRP and the other managed by LegacyUserManager,


each with different passwords.

Following we can see the structure of the sec$user_attributes


table, responsible for storing the tags (attributes) defined for users:
SQL> show table sec$user_attributes;
SEC$USER_NAME (RDB$USER) CHAR(31) CHARACTER SET UNICODE_FSS
Nullable
SEC$KEY (SEC$KEY) VARCHAR(31) CHARACTER SET UNICODE_FSS
Nullable
SEC$VALUE (SEC$VALUE) VARCHAR(255) CHARACTER SET
UNICODE_FSS Nullable
SEC$PLUGIN (RDB$PLUGIN) CHAR(31) CHARACTER SET UNICODE_FSS
Nullable

Where:
sec$user_name – stores the user name;
sec$key – stores the tag name;
sec$value – stores the tag’s value;
sec$plugin – stores the name of the user management plugin;

By using the select statement, you can retrieve information about


users. Below we have an example of the result of a select, run by the
SYSDBA or a user logged in with the rdb$admin role:
SELECT U.SEC$USER_NAME as LOGIN, A.SEC$KEY as TAG,
A.SEC$VALUE as "VALUE", U.SEC$PLUGIN "PLUGIN"
FROM SEC$USERS U
LEFT JOIN SEC$USER_ATTRIBUTES A
ON U.SEC$USER_NAME = A.SEC$USER_NAME
and U.SEC$PLUGIN = A.SEC$PLUGIN;

Example of the returned values:


LOGIN TAG VALUE PLUGIN
==================== =========== ========= ==================
SYSDBA <null> <null> Srp
SYSDBA <null> <null> Legacy_UserManager
ALBERT SPECIALTY math Srp
ALBERT DEGREE genius Srp
JOHN <null> <null> Srp

Below we can see the result of the same select, this time run by the
user Albert, logged in without the rdb$admin role, in other words,

Users in
Firebird 3
74

like a “normal” user. Note that only information about himself is


returned:
LOGIN TAG VALUE PLUGIN
==================== ============= ==================== ======
ALBERT SPECIALTY math Srp
ALBERT DEGREE genius Srp

Next, we can see the result of the same select, executed again by
user Albert, but this time logged in with the role rdb$admin:
LOGIN TAG VALUE PLUGIN
==================== =========== =========== ==================
SYSDBA <null> <null> Srp
SYSDBA <null> <null> Legacy_UserManager
ALBERT SPECIALTY math Srp
ALBERT DEGREE genius Srp
JOHN <null> <null> Srp

Did you notice that the result is exactly the same as when the
sysdba ran the statement? Assigning the role rdb$admin to a “normal”
user gives him the power of the SYSDBA.

 The column MON$SEC_DATABASE was


added to the table MON$DATABASE, and
returns the type of the security database being
used on the connection. Possible values are:

• Default: : security3.fdb;
• Self:: the database stores the users locally;
• Other: a specific non-default security
database;

Preparing a script to insert users into the new server

You cannot "migrate" the security database from previous


versions to Firebird 3. All existing users will have to be recreated in
Firebird 3!
For applications which authenticate users in the application and
always use the same user to login on Firebird, the task is simple,

Users in
Firebird 3
75

because there will be a small number of users to be created in the


Firebird 3 security database, perhaps only the SYSDBA and a few
more. But those who allow Firebird itself to authenticate users, have
to create dozens or hundreds of users.
Before migrating to Firebird 3, I recommend that you prepare a
user creation script. Now that Firebird allows managing users via
SQL, this task is much easier!
To get the list of all existing users in the older Firebird server,
you can use gsec, as in the example below:
C:\firebird25\bin>gsec -display -database
c:\firebird25\security2.fdb
user name uid gid admin full name
---------------------------------------------------------------
SYSDBA 0 0 Sql Server Administrator
JOHN 0 0
JOSEPH 0 0
MARY 0 0

Redirect gsec output to a text file, and edit this file to preserve
only the names of the users. Manually insert the CREATE USER
stataments, line by line, resulting in a script like this:
create user JOHN password ‘john_password’;
create user JOSEPH password ‘joseph_password’;
create user MARY password ‘mary_password’;
commit; -- Don’t forget to COMMIT!

Unfortunately (or rather, fortunately!), there's no way to recover


the passwords of existing users. If you know the passwords, you can
insert them manually in the script, or you can set a default password
and instruct users to change it later.
Another way to get the list of users in older versions of Firebird
is to connect directly to the security2.fdb file and select user names. The
problem is that in the Firebird 2.5, it is no longer possible to
connect to the security database that is in use. You can get around
that by stopping Firebird proccess and copying the security database
to another directory, and then connecting to the copy to retrieve the
users list through a select, which will also be responsible for
assembling the lines of script:
select 'create user ' || R.RDB$USER_NAME ||
' password ''' || cast(cast(trunc(rand() * 1000000000) as
integer) as varchar(10)) || '''' ||

Users in
Firebird 3
76
coalesce(' firstname ''' || R.RDB$FIRST_NAME || '''', '') ||
coalesce(' middlename ''' || R.RDB$MIDDLE_NAME || '''', '') ||
coalesce(' lastname ''' || R.RDB$LAST_NAME || '''', '') || ';'
from RDB$USERS R;

This method has the advantage of allowing you to extract the first
name, middle name, and last name, when this information exists. In
addition, you can assign each created user a new "random"
password. In the example, a numeric random password will be
generated for each user, by using the built-in function RAND(). The
result would be something like:
create user SYSDBA password '871892958' firstname 'Sql'
middlename 'Server' lastname 'Administrator';
create user JOHN password '671411450';
create user JOSEPH password '723543343';
create user MARY password '140440819';
commit; -- Don’t forget to COMMIT!

Remember to initialize the Firebird 3 security database, before


you run the script to create the users. Remove the SYSDBA user
from the generated script, since the security database has already
been initialized, and therefore the SYSDBA already exists.
Don't forget to issue a commit at the end of the script, so that users
will be actually created.

 The RAND and TRUNC functions do not exist


in very old versions of Firebird. In those versions,
you can use similar functions available through
UDFs.

 The user creation script can create users in the


default security database, in specific databases, or
even locally, inside an application database
configured to be its own security database.

 If you plan to use Firebird 3 with legacy


authentication, instead of the default SRP,
configure the server to use the Legacy_UserManager
plugin before you create users. Alternatively, you
can specify the plugin to use in each create user
statement of the script, with the clause using plugin

Users in
Firebird 3
77

Legacy_UserManager.

Firebird 3 Release Candidate 2 introduced a user creation script


(security_database.sql), which uses a method similar to that described
above, but acts in a slightly different way.
The security_database.sql script that comes with Firebird 3 works
this way:

1. In the older server, backup the security database (ie:


security2.fdb) using gbak, i.e.:
gbak -b -user SYSDBA -pas masterkey
{host/path}security2.fdb security2.gbk
2. Under Firebird 3, restore the backup:
gbak -c -user SYSDBA -pas masterkey security2.gbk
{host/path}security2-ods12.fdb
3. Run the user migration script:
isql -user sysdba -pas masterkey -i
security_database.sql {host/path}security2-ods12.fdb

Here is the user migration script with some comments about what
is being done:
set term ^;
execute BLOCK returns(USR varchar(31), PASSWD varchar(36))
as
declare variable FRST varchar(32);
declare variable MDDL varchar(32);
declare variable LST varchar(32);
declare variable ATTR varchar(4096);
declare variable SQL varchar(4096);
declare variable UID int;
declare variable GID int;
begin
-- Retrieve the fields related to the full name of the user
for select RDB$USER_NAME, RDB$FIRST_NAME, RDB$MIDDLE_NAME,
RDB$LAST_NAME,
RDB$UID, RDB$GID, UUID_TO_CHAR(GEN_UUID())
from RDB$USERS
where RDB$USER_NAME is not null and
upper(RDB$USER_NAME) != 'SYSDBA' – skip SYSDBA
into :USR, :FRST, :MDDL, :LST, :UID, :GID, :PASSWD
do
begin
-- Assembly the SQL statament for user creation
SQL = 'create or alter user ' || USR || ' password ''' ||
PASSWD || '''';
if (FRST is not null) then
SQL = SQL || ' firstname ''' || FRST || '''';
if (MDDL is not null) then

Users in
Firebird 3
78
SQL = SQL || ' middlename ''' || MDDL || '''';
if (LST is not null) then
SQL = SQL || ' lastname ''' || LST || '''';
SQL = SQL || ' active';
-- retrieve the GID if it exists
ATTR = '';
if (UID is not null) then
ATTR = 'uid=''' || UID || '''';
if (GID is not null) then
begin
if (char_length(ATTR) > 0) then
ATTR = ATTR || ', ';
ATTR = ATTR || 'gid=''' || GID || '''';
end
if (char_length(ATTR) > 0) then
begin
SQL = SQL || ' tags (' || ATTR || ')';
end
execute statement SQL; -- creates the user
suspend; -- returns the user and generated password
end
end^
commit ^
exit ^

The script reads the existing users on the old security database,
retrieving the names and attributes, and creates those same users in
Firebird 3, with a random password generated by the internal gen_uuid
function. When executed, the script will display the names of the
users and their generated passwords. Note that the script is
configured not to migrate the SYSDBA user.
Choose the whichever migration method best suits your needs.

Users in
Firebird 3
79

Protecting your data


Learn why the subject is much more complex than it seems, and why
the complete solution is beyond Firebird...

Protecting your data


80

The question "how to secure a database" against "thieves" or


"nosy" people has been controversial among Firebird users as long
as Firebird has existed.
Data security is a complex issue by itself and gets even worse
when the subject is an open source RDBMS. Anyone can download
the Firebird source, remove all the code that provides data security,
and rebuild it, generating a "custom" pirate Firebird version! How
can you secure your data?
Data security is not directly in the subject of this book, but I
decided to write this chapter because security is so important
Firebird 3 includes improvements in security; among others
default network encryption, and local users, which provide more
independence when you have several databases in the same server.
However, nothing will protect you, if your server’s environment
doesn’t offer the minimum security required in an enterprise
environment.
Unfortunately, Firebird is often installed in unprotected
environments, especially in small businesses, where there is no
security policy at all, not even an IT professional to manage the
server and the network. For these users, ensuring the security of the
data is virtually impossible unless they understand the risks. Without
some concern about configuring users and access rights to the
server’s directories and files, there is no way that any file or
information can be safe.
To make things worse, in the past years, some "hacks" have
appeared in the Firebird community that give the illusion of
security, especially when the goal is avoiding giving the SYSDBA
user access to the database. For example, creating a ROLE named
SYSDBA to prevent the SYSDBA user from connecting to the
database is totally ineffective! Anyone who has physical access to
the database file can remove that ROLE by editing the file with a
hexadecimal editor. Another way to work around the SYSDBA role
is to copy the database file to private server, use gstat to learn the
name of the owner of the database and create a user with that name
on the private server. Furthermore, Firebird 3 does not allow direct
updates to the system tables, so creating a role named SYSDBA is
impossible in Firebird 3.
Another common method to deny the user SYSDBA access to
databases is creating a database trigger that checks the name of the

Protecting your data


81

user logging in. If the name is SYSDBA or any other undesirable


user, the trigger breaks the connection immediately. Unfortunately,
connecting to a database without firing the database trigger is simple.
In isql using the parameter -nod, turns off database triggers, making
such “protection” useless.
None of these "hacks" prevents anyone who has physical access
to the database from connecting to it with an embedded connection,
which does not validate users and passwords.
These "hacks" may fool some novices and the idly curious. They
cannot prevent experienced users from getting their hands on the
contents of the database.

Creating a secure environment

The first tip seems silly, but I have seen an enormous number of
servers where whoever installed Firebird didn't even bother to
change the default SYSBDA password. So let me scream out loud:
never install the Firebird using the masterkey password! Any
idiot with internet access knows that password.
Furthermore, a Firebird database file must be accessible only
by the Firebird process and the system administrator. Right!
Client applications do not need physical access to the database.
"Ordinary" users do not need and should not have physical access
rights to the database file. Restricting access prevents copying the
database for attack.
Therefore, as soon as you have configured a Firebird server, you
should restrict access to the database files, preventing thieves from
copying them.
You must protect your backup files too! There's no point in
restricting access to the database files if you store your
backups in public directories, or write them to CDs/DVDs left
lying on your desk.

Don't give ordinary users access to the server console, and


ensure that they don't know the password for the Operating
System administrator!

Protecting your data


82

Encrypting the database file

Database file encryption is a new Firebird 3 feature. By using


special crypt plug-ins, you cause Firebird to encrypt data, blob and
indexes pages that it stores.
However, to guarantee the effectiveness of any protection via
encryption, in addition to the algorithm used, you must protect
encryption key! A "thief" can steal the database file, but will not be
able to access the information, if he does not have both the crypt
plug-in and the encryption key.

 As important as the strength of the cryptographic


algorithm, is how it handles the storage of the key.
Below are some possible scenarios, and their
drawbacks regarding the key storage:

Hardcode the key in the crypt plug-in: Hacker steals the


plug-in library, gets the key, and has your data.

Hardcode the key in the client applications: Hacker


decodes the application executable and gets access
to the key, and has your data.

Key stored in external device (i.e. pen drive, usb token, etc):
Hacker steals the device and the plug-in library,
and has your data.

Use external encryption device (ex: HSM): Hacker steals


the device and the plug-in library and has your
data.

Manually enter the key each time the crypt plug-in is


loaded: Someone must be available whenever the
server restarts. Chances of mistyping the key.

By design, Firebird 3 does not provide a "default" plug-in for


database encryption. This decision was made because the project is
open source. If the code of an encryption plug-in is publicly

Protecting your data


83

available, anyone can download it, study it, figure out how it handles
keys, etc. making it ineffective, or giving a false sense of security.
Therefore, developers should create their own encryption plug-
ins, making every effort to handle the key securely. In a near future,
companies will offer encryption plug-in solutions, making the life
easier for those who do not want to create their own.
Although a Firebird installation doesn't create a default
encryption plug-in, it includes a sample plug-in in C++ called
DbCrypt.cpp, located in the examples\dbcrypt subdirectory.
The Firebird 3 Crypto API includes methods for providing the
cryptographic key via the client application, sending the key from the
client to Firebird across the network in an encrypted connection,
obviously. An example of this technique can be seen in the
CryptKeyHolder.cpp file, also available in the examples\dbcrypt directory.
Note, however, that an experienced hacker could create a custom
version of Firebird that collects passwords and keys in a file or even
in the firebird.log.
Firebird 3 includes special commands that allow you to encrypt
and decrypt databases "on the fly", without exclusive access to the
database file. When encryption is activated, Firebird will start
encrypting the database in background. The process is transparent
to end users.

 Remember that a backup of an encrypted


database, made by gbak, generates an unencrypted
backup file! So, pay attention to where you put
your backup files! Consider using an encryption
tool on your backup files.

This book cannot teach how to develop an encryption plug-in or


even how to use one. The Release Notes and previously mentioned
examples are your best source of information for managing
encryption.

Protecting your data


84

Conclusion

To guarantee that no one can steal your data, follow these steps:

1. Never use the masterkey password.


2. Restrict the physical access to the database file to Firebird
and the system administrator.
3. Do not leave the backup files publicly available.
4. Encrypt your data with an encryption plugin and make
sure that the thieves cannot access the plug-in library
and the encryption key.

If you take step 4 seriously, you can keep malicious people out of
your data.

Protecting your data


85

Wire Protocol
Enhancements
Traffic encryption, compression and optimizations

Wire Protocol Enhancements


86

Traffic encryption

In older versions of Firebird, data was sent between client and


server over the network unencrypted, with no privacy protection.
Network sniffers allowed anyone to capture the network packets and
extract information from them. Some people work around that
weakness by creating an encrypted VPN using software like
Zebedee (www.winton.org.uk/zebedee). The good news is that
Firebird 3 includes native network traffic encryption!
A new Firebird 3 installation has traffic encryption enabled by
default, and I recommend leaving it enabled.
The parameter WireCrypt in firebird.conf controls encryption. The
encryption can be enabled/disabled using that parameter, which
accepts three possible values:

• Required – requires encrypted connection.


• Enabled – allows encrypted connection.
• Disabled – rejects encrypted connection.

The decision whether the connection is established with


encryption depends on the combination of the WireCrypt parameters
on the server and the client sides.
The following table shows the possible combinations:

Client Server
Disabled Enabled Required
Disabled Disabled Disabled Rejected
Enabled Disabled Active Active
Required Rejected Active Active
Encryption status based on the configuration of the WireCrypt parameter in client
and server. The highlighted options are the default.

The default value of WireCrypt in the server is Required, which


causes Firebird to reject all unencrypted connection attempts. At the
client side, the default is Enabled, which allows the client to connect
to servers configured with Required or Enabled encryption settings.

Wire Protocol Enhancements


87

The default encryption method used by Firebird 3 is RC4


(wikipedia.org/wiki/RC4). Configuring the WireCryptPlugin
parameter to work with other plug-ins allow the use of different
encryption algorithms. The default value for the WireCryptPlugin
parameter is Arc4, meaning that the encryption is based on the
Alleged RC4 standard, which is the encryption used in the SSL
connections. You can develop your own encryption plug-in, or use
third party plug-ins when they become available.
In RC4, the cryptographic key is defined automatically at the
handshake between the client and the server, so you don’t need to
worry about the key.

 There is no data encryption between the client


application and the Firebird process when access
to the database is through the XNET Protocol.
XNET is available only on Windows and only for
local connections.

You may wonder if the use of encryption in data traffic degrade


the performance of client/server communication, but with the
processing power of modern computers, the difference is not
noticeable.
The following chart compares the speed of different encryption
algorithms. The RC4 is one of the most efficient.

Source: http://www.javamex.com/tutorials/cryptography/ciphers.shtml

Wire Protocol Enhancements


88

How effective is the encryption? Below we can see the effect of


encryption the string “Firebird Developers Day” using RC4 and the
key “essaeminhasenha”.

Traffic compression

Firebird 3 can compress the data exchanged between the client


and the server. The client side activates the compression at
connection time if the parameter WireCompression is set to true.
The default of Firebird 3 is to disable wire compression.
When compression is enabled, Firebird 3 uses the Zlib library
(www.zlib.net), which must be available on both the server and the
client computer. The Firebird installer includes the correct zlib
library.

 Compressing data consumes CPU resources.


Enable compression for access between client and
server through a WAN (e.g. Internet) but not for

Wire Protocol Enhancements


89

high-speed local connections. In local network


connections, the speed gain from compression
may be imperceptible, or even negative! When
data being transferred is small, the speed gain is
likely to be small.

Tests conducted with a client located in Brazil, accessing a


remote Firebird 3 server hosted in the United States via the Internet
showed a significant speed gain using compression in fetchalls
returning a large number of records, as we see later in this chapter.

 The type of information being transferred affects


the efficiency of data compression. Textual data
has a high rate of compression. Other data, like
jpeg images, zip files, etc. are already compressed.
There is no benefit from compressing previously
compressed data.

The client activates compression during the connection request.


A client application can connect to a number of different servers,
some of them on the local network, others in remote locations,
accessed via the Internet or satellite link. If the client application
could choose whether the compression should be active in a given
connection, it could optimize performance for each type of
connection.
Unfortunately, setting the WireCompression parameter in the
firebird.conf file affects all of the client's connections. However, the
client can enable wire compression at “connection level”, using the
DPB (Database Parameter Block). Some of Delphi’s native access
components, like IBObjects, may allow the client to configure the
DPB. Since per-database compression is of significant benefit in
Firebird 3, the authors of interface components may create more
practical ways to enabled/disable compression on a connection by
connection basis.
For users of Delphi and IBObjects components, the method
below configures the connection with compression enabled,
through the onCustomizeDPB event of the IB_Connection component:

Wire Protocol Enhancements


90
procedure TForm1.IB_Connection1CustomizeDPB(Sender:
TIB_Connection;
var ABufPtr: Integer; var ABuffer: array of Byte);
const isc_dpb_config = 87;
begin
Sender.BuildDPBStr(ABufPtr, ABuffer, isc_dpb_config,
'WireCompression=true');
end;

 When using isql to connect to a Firebird 3 server,


the show version command includes the state of
encryption and compression for the connection. If
the end of the result displays a C, it means that
encryption is active. The presence of a Z means
that compression is active, as shown in the
example below:
Firebird/Windows/Intel/i386 (remote server), version
"WI-V3.0.0.32366 Firebird 3.0 Release Candidate
2/tcp (B153max)/P13:CZ

Enhancements for usage in high latency networks

The Firebird wire protocol is “chatty”, generating a lot of


communication between the client and the server during the
execution of tasks. In a local network, the chatter has little effect on
performance. In high latency networks (e.g. the Internet), the effect
can be striking.

 In networks, the term latency determines the time


taken for a packet to go from source to
destination and back (roundtrip).

Accessing a remote Firebird server over the Internet is much


slower than on a local network. Firebird has a reputation as a slow
database when used over the Internet.
Once again, using VPNs works around the problem gaining a bit
of performance when data compression is enabled. An n-tier
application architecture is a more robust solution. However, legacy
systems developed using the client/server model are expensive to
redesign and implement.

Wire Protocol Enhancements


91

Firebird 2.1 brought improvements to the communication


protocol, decreasing the number of roundtrips by up to 50%,
leaving it on average 30% faster on high latency networks. Even so,
performance was far from ideal.
During the ninth edition of the Brazilian Firebird Developers
Day, attendees donated funds to sponsor improvements to
Firebird’s communication protocol. Dmitry Yemanov, chief of the
Firebird development team, undertook to implement them, and the
result can be seen in Firebird 3.0
The enhancements were implemented in two situations:

• Null data is transferred in bitmaps. Previously, nulls


occupied 4 bytes + the field’s declared size! Fetching
groups of records containing many null fields is much
faster in Firebird 3. The effect is most obvious when the
null fields are declared as long strings.
• The pre-fetch algorithm uses the actual size of the data
being sent, rather than the declared record size. This
change makes communication packets more dense
because they're not full of blank space.

Unfortunately, in some cases the protocol remains slow. Those


cases often involve transferring blobs. Firebird cannot transfer the
contents of a blob with other data. To transfer the contents of a blob
requires a separate call, which can generate up to 3 extra roundtrips.
The following chart shows the time difference in a fetchall
returning 10,000 records from a table with customer data. The black
bars represent customers records collected from real word database,
where several of the fields were null. The gray bars represent the
same customer records, with the null fields filled with random data.

Wire Protocol Enhancements


92

The chart scale is seconds.


The difference in performance is clear. Firebird 3 using
compression (WireCompression) and setting the TcpRemoteBufferSize
parameter as 32 KB is significantly faster. You must configure
TcpRemoteBufferSize in the server’s firebird.conf.

 Use the TcpRemoteBufferSize parameter with


caution. Increasing the TCP/IP buffer size
exchanges more data in one packet. That can be
beneficial when a large volume of data is being
transferred. When the amount of data exchanged
is small, a large TcpRemoteBufferSize wastes
resources.. The default value for this parameter is
8192 bytes (8 KB).

The next chart compares the performance of Firebird and


MySQL. The test was done with MySQL 5.3.6, containing the same
customer table used in Firebird, with exactly the same data. It shows
that Firebird 3 was faster than MySQL in all four tests when
fetching 10,000 records.

Wire Protocol Enhancements


93

The situation changes if you add blob fields to the transferred


data. With textual blobs, you can improve performance if you know
that none of the blobs is longer than 32765 characters by returning
the blob cast as a varchar (32765).
The chart below shows the time taken to fetch 1,000 records
where one of the fields was a text blob. Even when casting the blob to
a varchar in Firebird, MySQL is still faster fetching data containing
blobs. Null Blobs doesn’t affect the performance, since they have no
content.

Wire Protocol Enhancements


94

These charts were copied from my session at the 12th Firebird


Developers Day conference, named "Using Firebird in High Latency
Networks". The environment used for the tests is described in the
following picture.

Wire Protocol Enhancements


95

Connection strings
Learn how to specify the transport protocol, ports, and service name
and how to use IPv6 address, etc. to establish a connection.

Connection strings
96

Legacy syntax

The connection string indicates to Firebird what transport protocol


to use in communication between the server and the client (e.g. local,
xnet, tcp/ip, netbeui/wnet), what database to open (using an alias or the
database file path), and the port or service name of the desired
Firebird server.
Developers using IDEs like Delphi may not be familiar with the
connection string syntax. Connection components offer individual
properties that build up a connection string (server, path, port, etc.).
Internally, the component assembles the connection string based on the
values of these properties, and then establishes the connection.
The syntax of a connection string is:

For TCP/IP (INET):


<host> [ / <port>] : <database file path or alias>

For NetBEUI (WNET):


\\ <host> [ @ <port | service name>] \ <DB file path or alias>

The path to the database file follows the rules of the host
operating system. For example, on posix systems, the path is case
sensitive.
Here are several examples of connection strings, some of them taken
from the Firebird 3 Release Notes:

TCP/IP connections specifying the database file:


192.168.0.11:/db/mydb.fdb
192.168.0.11:C:\db\mydb.fdb
myserver:C:\db\mydb.fdb
localhost:/db/mydb.fdb
www.remotesite.com:/databases/mydb.fdb

TCP/IP connections specifying a database alias:


192.168.0.11:mydb
myserver:mydb
localhost:mydb

Connection strings
97

TCP/IP connections specifying a non-default port (port 3051):


192.168.0.11/3051:C:\db\mydb.fdb
192.168.0.11/3051:mydb
myserver/3051:/db/mydb.fdb
localhost/3051:/db/mydb.fdb
myserver/3051:mydb
localhost/3051:mydb

TCP/IP connections specifying a non-default Firebird service


name (fb_db):
192.168.0.11/fb_db:C:\db\mydb.fdb
192.168.0.11/fb_db:mydb
localhost/fb_db:/db/mydb.fdb
myserver/fb_db:/db/mydb.fdb
myserver/fb_db:mydb
localhost/fb_db:mydb

 For TCP/IP connections, you can specify the host


machine by IP address or host name. Use the IP
address only if you are certain that it is fixed,
otherwise, each time the IP address changes, you
must update your connection string.

Using the host name is preferred, but it can also


fail if the machine cannot translate the address to
its IP number due to DNS failure, bad network
configuration, etc.

NetBeui connections are available only on Windows:


\\myserver\C:\db\mydb.fdb
\\myserver@fb_db\C:\db\mydb.fdb
\\myserver@3051\C:\db\mydb.fdb

Local connections:
/db/mydb.fdb
C:\db\mydb.fdb
mydb

Connection strings
98

 Firebird automatically adjusts the slash (/) and


backlash (\) in the database file path to follow the
server operating system rules. However, for the
sake of clarity, you should use the server operating
system rules.

URL based syntax

Firebird 3 accepts the legacy syntax for connection strings, and


introduces a new syntax based on the URL format:
[ <protocol> : // [ <host> [ : <port> ] ] ] / <database file
path or alias>
<protocol> ::= INET | WNET | XNET

Where:

INET = TCP/IP
WNET = NetBeui (named pipes)
XNET = Local connection via shared memory

 The XNET and WNET (NetBeui) protocol are


available only on Windows.

Using the URL syntax, you can specify exactly how you want to
make the connection with Firebird, as in the following examples:

TCP/IP connections using the database file path:


inet://192.168.0.11//db/mydb.fdb
inet://192.168.0.11/C:\db\mydb.fdb
inet://myserver/C:\db\mydb.fdb
inet://localhost//db/mydb.fdb
inet://www.remotesite.com:/databases/mydb.fdb

TCP/IP connections using an alias:


inet://192.168.0.11/mydb
inet://myserver/mydb
inet://localhost/mydb

Connection strings
99

TCP/IP connections specifying port 3051:


inet://192.168.0.11:3051/C:\db\mydb.fdb
inet://192.168.0.11:3051/mydb
inet://myserver:3051//db/mydb.fdb
inet://localhost:3051//db/mydb.fdb
inet://myserver:3051/mydb
inet://localhost:3051/mydb

TCP/IP connections specifying the service name:


inet://192.168.0.11:fb_db/C:\db\mydb.fdb
inet://192.168.0.11:fb_db/mydb
inet://localhost:fb_db//db/mydb.fdb
inet://myserver:fb_db//db/mydb.fdb
inet://myserver:fb_db/mydb
inet://localhost:fb_db/mydb

NetBeui remote connections:


wnet://myserver/C:\db\mydb.fdb
wnet://myserver:fb_db/C:\db\mydb.fdb
wnet://myserver:3051/C:\db\mydb.fdb

NetBEUI local connections:


wnet://C:\db\mydb.fdb

Loopback (localhost) connections, via TCP/IP:


inet:///db/mydb.fdb
inet://c:\db\mydb.fdb

Pay attention to how Firebird understands a connection string


without a host name specified, as in the example below:
c:\databases\base.fdb

In this case, Firebird 3 assumes that you want to make a local


connection, since you did not specify a host name. However, it does
not specify whether you want to make an embedded, xnet or a
loopback connection.
The firebird.conf file has a list of providers in the following order:
Remote, Engine12 (embedded) and loopback. When Firebird receive a

Connection strings
100

connection request using the string above, it runs through the list of
providers, from left to right, until it finds a provider that recognizes
and creates the connection.
In this example, the first provider (Remote) fails, since there is no
host name in the string. The connection request then goes to the
second provider (Engine12), which establishes an embedded
connection.
However, what do you do to make an xnet connection rather
than an embedded connection? In that case, you must specify the
xnet protocol in the connection string:
xnet://c:\databases\base.fdb

 You might ask why you would want to use xnet


instead of embedded for a local connection. One
possible scenario is that Firebird is running as
SuperServer with an active connection to that
database. In that case, the embedded connection
to the database fails because Firebird already has
the database file open in exclusive mode.
However, the xnet connection works since it will
connect through the running SuperServer.

 In isql, the show version command is an easy way to


determine what kind of connection has been
established:
SQL>show version;
Firebird/Windows/Intel/i386 (remote server),
version "WI-V3.0.0.32136 Firebird 3.0 Release
Candidate 1/tcp (myserver)/P13:C"
SQL>show version;
Firebird/Windows/Intel/i386 (remote server),
version "WI-V3.0.0.32136 Firebird 3.0 Release
Candidate 1/XNet (myserver)/P13"

Another way to show the connection type is to


read the MON$REMOTE_PROTOCOL field of
the mon$attachments monitoring table.

Connection strings
101

IPv6 support

Firebird 3 supports TCP/IP connections using IPv6 addresses.


However, addresses in this format use the “:” symbol as separator.
Firebird uses the ":" symbol to separate the host name from the alias
or database path in the connection string.
To resolve this ambiguity, the IPv6 address in the connection string
should be enclosed in square brackets:
SQL>connect '[2014:2015:::1]:employee';
SQL>connect '[1234:1234:1234::5]/3051:c:\DB\my.fdb';

 A new parameter, IPv6V6Only, is available in


firebird.conf. It requires Firebird to accept only IPv6
TCP/IP connections. The default is set to false,
meaning that Firebird accepts both IPv4 and IPv6.

Connection strings
102

Firebird 3 and
legacy applications
What should you check in your applications, so they work well with
Firebird 3?

Firebird 3 and legacy applications


103

Firebird 3 introduces a new C++ object oriented API, while


maintaining full compatibility with the legacy API. You can use the
new client library with legacy applications unchanged, since the
library still supports all the old API calls.
Nevertheless, you should examine some aspects of existing
applications to avoid potential problems.

.NET applications

When this book was written, the Firebird .NET Provider did not
yet support the new communication protocol encryption or SRP
authentication, which are the default on Firebird 3.
The .NET Provider implements the Firebird communication
protocol natively. It does not depend on the client library (fbclient) to
communicate with the server. Therefore, it needs its own
implementation of the updated wire protocol to support new
features like encryption, compression and SRP authentication.
To connect to Firebird 3 with an application using the .NET
Provider, you need to set the following parameters in firebird.conf:
WireCrypt = enabled

The default value for the WireCrypt parameter is required, forcing


all clients to use encrypted connections. Changing the value to
enabled, allows Firebird to accept unencrypted connections.
AuthServer = Legacy_Auth, Srp, Win_Sspi

The default value for AuthServer is SRP. However, for the .NET
Provider to connect to Firebird 3, you must change to the "legacy"
authentication, even though it is less secure. You can set the
AuthServer configuration for specific databases using the
databases.conf file.

 You can also configure both AuthServer and


AuthClient per connection using the DPB (Database
Parameter Block) in the connection request.

Firebird 3 and legacy applications


104

Jaybird applications

Jaybird is the JDBC driver for Java applications. Like .NET


Provider, Jaybird has a native implementation of the Firebird
communication protocol in pure Java, without relying on fbclient –
the Firebird client library – although you can configure Jaybird to
connect through the Firebird client library.
When this book was written, the native implementation of
Firebird communication protocol in Jaybird did not yet support SRP
authentication, compression, or network encryption. According to
Mark Rotteveel, the developer of the driver, network encryption will
be available in Jaybird 3.1, or possibly version 3.0, which will also
include support for SRP authentication. So far, there is no schedule
for support of data compression.
Therefore, Jaybird 2.2 users of the native implementation of the
communication protocol must configure Firebird 3 to use legacy
authentication (LegacyAuth) and not to require encryption of data
traffic (WireCrypt).
WireCrypt = Enabled
AuthServer = Srp, Win_Sspi, Legacy_Auth
UserManager = Srp, Legacy_UserManager

For more information about using Jaybird with Firebird 3, see


github.com/FirebirdSQL/jaybird/wiki/Jaybird-and-Firebird-3.0-
beta-2 or goo.gl/yp2ZwW. Find the Jaybird FAQ on
goo.gl/9GQ6NC or
www.firebirdsql.org/file/documentation/drivers_documentation/ja
va/faq.html.

 You can configure Jaybird to use the Firebird


client library (fbclient), instead of its native
implementation of the communication protocol.
That way, Jaybird can connect to Firebird 3 with
SRP, encrypted connections, and data
compression. However, the search path for the
operating system where Jaybird is used must
include fbclient, and the java library path must
include the jaybird22 or jaybird22_x64 libraries.

Firebird 3 and legacy applications


105

For Jaybird to use fbclient, the JDBC connection


string must mention "native", following this
format:
jdbc:firebirdsql:native:host[/port]:<database>

Logical data type (Boolean )

Firebird 3 introduces a new data type to represent logical values


of True and False called Boolean. Before defining Boolean fields in
your databases, make sure that the languages and interface tools
used in your applications can support it. Otherwise, your application
cannot “understand” the values stored in such fields!
For example, applications using the deprecated BDE can never
recognize and use this new Firebird data type.

Connecting to Firebird 3 with an old client library


(fbclient)

You should use the same version of the Firebird client library as
the installed server so you can take advantage of new features. If
you need to connect to Firebird 3 using an "old" fbclient, you must
configure the firebird.conf file to use legacy authentication and to
accept connections without encryption:
WireCrypt = enabled
AuthServer = Legacy_Auth, Srp, Win_Sspi

Remember that the legacy authentication is less secure than the


SRP authentication.

Query performance

After switching to a new version of Firebird, queries that


previously performed well may slow down - although rare, it can
happen! The main reasons for performance degradation are:

Firebird 3 and legacy applications


106

1. Changed index statistics. You must backup and restore


databases to migrate to Firebird 3, which updates all
index statistics. Changing index statistics can cause the
optimizer to choose a different access plan for queries.
Sometimes, the optimizer makes mistakes when choosing
the best plan, degrading performance.
2. Optimizer changes. Every new Firebird includes changes
to the optimizer making it generally better. However, in
some cases, these improvements produce "collateral
damage" and the optimizer chooses an inefficient plan
for a few queries.

If you see worse performance on some queries, report the defect


in the Firebird tracker, tracker.firebirdsql.org. The core developers
analyze problems reported there and determine whether they can
improve the optimizer logic. As a short-term solution, you can try to
force a plan, or even use some tricks to fool the optimizer, forcing it
to choose a different plan.

 Firebird does not update the index statistics


automatically, but only when creating or
rebuilding an index, or when you run the SET
STATISTICS command. Many applications fail to
update the indexes statistics. As a result, the
statistics may not reflect current data. Developers
should keep that in mind, and implement routines
in applications to recalculate the statistics from
time to time. That said, after a gbak restore index
statistics reflect the actual data distribution.

Reserved words

The words below are reserved in Firebird 3. This means that if


you used those words as the names of tables, fields, or other objects,
or if you used them as variable names in stored procedures or
triggers, you should rename the objects to a non-reserved word, or
reference them using “ ” (double quotes) making them quoted
identifiers.

Firebird 3 and legacy applications


107

BOOLEAN REGR_AVGX SCROLL


CORR REGR_AVGY SQLSTATE
COVAR_POP REGR_COUNT STDDEV_POP
COVAR_SAMP REGR_INTERCEPT STDDEV_SAMP
DELETING* REGR_R2 TRUE
DETERMINISTIC REGR_SLOPE UNKNOWN
FALSE REGR_SXX UPDATING*
INSERTING* REGR_SXY VAR_POP
OFFSET REGR_SYY VAR_SAMP
OVER RETURN
RDB$RECORD_VERSION ROW

The words marked with * were already recognized by Firebird,


but were not reserved.

 Only Dialect 3 supports the use of double quotes


in identifiers. If your database still uses Dialect 1,
you must rename those identifiers using non-
reserved words.

Note that if your database includes newly reserved words,


backing it up and restoring it with Firebird 3 does not produce error
messages. The gbak restore recognizes table and column names that
are reserved words and changes them to quoted identifiers.
However, triggers, stored procedures, validity constraints, and even view
definitions may contain Firebird 3 reserved words that are not
detected or corrected by gbak. When an application executes queries
referencing database objects whose names have become reserved
words, or when the DBA tries to recompile triggers and procedures
that reference reserved words in their bodies, Firebird reports
errors. The easiest solution to the problem is to change the queries,
procedures and triggers, enclosing the reserved words in double
quotes. Note that doing so, the names become case sensitive.

Firebird 3 and legacy applications


108

 When migrating to Firebird 3, notice whether your


database refers to any word that became reserved
in Firebird 3. An easy way to verify this is to
extract the metadata from the database on the old
server into a script, and use the script to create a
new database in Firebird 3, correcting any errors
reported. Read the chapter "Migrating an existing
database" for more information.

Manipulating the System tables (RDB$...)

All Firebird databases include system tables. You can identify


them by their names, which start with RDB$. They store
information about the metadata of the database, i.e. names of tables,
fields, types, sizes, procedures, triggers, etc. Before Firebird 3, those
tables were active, meaning that an update to them changed the
database. Older Firebird utilities performed metadata changes
directly. Some applications did the same, which was quite
dangerous. The system tables were not hardened against ill-
considered changes; those changes could corrupt the database.
Firebird 3 no longer allows direct manipulation of system tables!
If your application performs updates, inserts, or deletes on system
tables, you must change it to use Data Definition Language
statements. SQL DDL cannot express most illegal changes and the
DDL implementation catches other errors.
With the expansion of SQL DDL statements in Firebird 3,
virtually all operations that could be done only through direct
changes to the system tables can now be done in "official" ways, via
SQL DDL.
The only exceptions are the fields that store the source code of
procedures and triggers, located in the RDB$PROCEDURES and
RDB$TRIGGERS tables, respectively. Application developers often
want to hide the human readable version of their procedures and
triggers to protect their intellectual property.SQL DDL offers no
way to delete or hide that source code. As an accommodation to
that business need, Firebird 3 allows you to manipulate those two
fields.

Firebird 3 and legacy applications


109

 Some may wonder how the procedures and


triggers continue to work after having their source
code deleted. To create a procedure or trigger,
Firebird compiles the source code into a language
called BLR (Binary Language Representation) and
stores the compiled code in another column in the
RDB$PROCEDURES or RDB$TRIGGERS
table. Firebird does not depend on the source to
execute procedures or triggers.

These scripts delete the source code of the user defined procedures
and triggers of a database.
update RDB$PROCEDURES
set RDB$PROCEDURE_SOURCE = null
where ((RDB$SYSTEM_FLAG = 0) or (RDB$SYSTEM_FLAG is null));
commit;
update RDB$TRIGGERS T
set T.RDB$TRIGGER_SOURCE = null
where ((T.RDB$SYSTEM_FLAG = 0) or (T.RDB$SYSTEM_FLAG is null))
and
not exists(select C.RDB$TRIGGER_NAME
from RDB$CHECK_CONSTRAINTS C
where C.RDB$TRIGGER_NAME =
T.RDB$TRIGGER_NAME);
commit;

 Some users also deleted the source code of views,


manipulating the rdb$relations table directly. You
can no longer delete the source of views.

 Be very careful when deleting the code of


procedures and triggers. If the source for your
objects is not available in at least one reference
database or in SQL scripts, you'll have trouble
when you need to change them.

Firebird 3 and legacy applications


110

Testing application’s queries

Changing from one Firebird version to another can cause


problems with existing queries, ranging from bad performance to
total failure when queries use newly reserved words or incorrect
SQL that had been accepted.
An application often includes of hundreds of queries, from
browsing information on the screen, to creating reports, to data
entry and validation.
FBScanner from IBSurgeon is a tool that can help you check that
existing applications work with Firebird 3. It allows you to log
queries on your current version of Firebird and test them on the
new Firebird server.
FBScanner is part of the HQBird package of analysis tools that
monitor and optimize Firebird servers. It works as a proxy between
the clients and the server, logging all communication. You can
install FBScanner while using your current version of Firebird;
configure it to monitor the communication and to store queries in a
log database. After a few days of monitoring, the log database
contains a large number of queries used in your applications.
FBScanner allows you to execute the queries you have logged.
Using the log database, you can set up a test Firebird 3 server,
restore the original database to it, and use the FBScanner Log Analyzer
to run the logged queries. FBScanner displays the result in a grid,
including the performance information, so you can compare the
currently execution time with what you got in the old server, and
compare the old PLAN to the new PLAN. The information in the
grid can be exported to an Excel spreadsheet.

 When comparing the execution time of queries,


keep in mind that external factors influence the
result. One consideration is whether the caches of
operating system or Firebird are "cold" or "hot".
Obviously, queries performed when most
information is already cached are faster than when
retrieving data from disk.

Firebird 3 and legacy applications


111

You can download a trial version of HQBird and its User Guide
from the IBSurgeon official site www.ibsurgeon.com.

 To use FBScanner for logging communication to


a Firebird Server 3, you must enable legacy
authentication and turn off wire encryption.
FBScanner can also be used as a Trace API plug-
in, instead of being installed as a proxy.

Using mon$attachments to get the number of active


connections

Many applications use the mon$attachments table to retrieve the


number of users and information about active connections, like
client IP, application process name, etc.
If you use information from mon$attachments with Firebird
SuperServer in Firebird 3, be aware that system connections created
by Firebird itself are included in the monitoring tables.
In this example, the select ran on a database with a single user
logged on:
isql employee
Database: employee, User: SYSDBA
SQL> select count(*) from mon$attachments;
COUNT
========
3
SQL> select mon$user, mon$system_flag from mon$attachments;
MON$USER MON$SYSTEM_FLAG
=============================== ===============
SYSDBA 0
Cache Writer 1
Garbage Collector 1

Even though there's only one user connected to the database, the
count returned 3. In the second select, you see that the other two
connections have the field mon$system_flag set to 1, indicating that
they are are system connections, created by Firebird itself and not
by a regular user.

Firebird 3 and legacy applications


112

To get the list or the number of active users at any given time,
use the command below:
SQL> select count(*) from mon$attachments
where mon$system_flag = 0;
COUNT
========
1

Default cache size for Classic/SuperClassic

In previous versions of Firebird, the default value for the page


cache for Classic/SuperClassic was 75 pages. In Firebird 3, this value
changed to 256. This means that more memory will be used for
cache by Classic/SuperClassic, for databases that doesn’t has the buffers
value explicitly defined. Make sure it will not consume too much
memory on the server.
You can define the buffer size explicitly in each database using
the gfix -buffers, or set it globally in firebird.conf, using the parameter
DefaultDbCachePages.

Mixing implicit and explicit joins

Firebird 3 will not accept certain “mixes” of implicit and explicit


joins in a select, depending on how one references the other fields. So,
some queries that ran before (even returning incorrect results) will
not execute in Firebird 3. Look at the following example:
select *
from TA, TB
left join TC on TA.COL1 = TC.COL1
where TA.COL2 = TB.COL2

The explicit join references a column from the implicit join. In


older Firebird versions, such query would fail or return an incorrect
result set. Firebird 3 will not execute this query.
More details can be seen at
tracker.firebirdsql.org/browse/CORE-2812.
To solve the problem, rewrite the select using explicit joins.

Firebird 3 and legacy applications


113

Count() now returns a BIGINT

The count() aggregator now returns a 64bits integer (BIGINT)


instead of a 32bits integer. If the database has procedures, triggers or
blocks of code where the result of count() is assigned to a variable, make
sure that the variable is declared as bigint, to avoid overflows.
If the applications have persistent fields declared for queries
containing count(), make sure to update the persistent field
declaration to a type which supports 64 bits values. The same
applies to local variables receiving the result of a count().

Firebird 3 and legacy applications


114

Appendix

Appendix
115

Macros
Firebird provides a series of macros to simplify references to
directories in the configuration files. The syntax is $(macro_name).

$(root)
Root installation directory of Firebird.

$(install)
Directory where Firebird was installed. Initially, $(install) and $(root)
are the same. However, $(root) can be overridden by setting the
FIREBIRD environment variable.

$(this)
Directory where the current configuration file is stored.

$(dir_conf)
Directory where firebird.conf and databases.conf are located.

$(dir_secdb)
Directory where the security database is located.

$(dir_plugins)
Directory where plug-ins are located.

$(dir_udf)
Default directory for UDFs.

$(dir_sample)
Directory where sample files are stored.

$(dir_sampledb)
Directory where the sample database (employee.fdb) is stored.

$(dir_intl)
Directory where the internationalization modules are located.

$(dir_msg)
Directory where the message file, firebird.msg, is stored. Normally this
is the same as $(root), but can be overridden by the FIREBIRD_MSG
environment variable.

Appendix
116

Note that you can include one configuration file inside another,
using the “include” directive, i.e.:
include my_files.conf

The relative path on the include clause starts from the directory
where the configuration file containing the include directive is stored.
The wildcards (* and ?) are recognized by the include directive,
allowing you to include files matching the mask, for example:
include $(dir_plugins)/config/*.conf

The above example would include all the files with the .conf
extension, located in the config subdirectory of the plug-ins directory.

 All parameters that were formerly expressed in


bytes can now be expressed in kilobytes, megabytes
or gigabytes a K, M and G – in upper case.

Configuration entries

The aliases.conf file of earlier Firebird versions is now called


databases.conf. With it, you can configure a number of parameters,
beyond just an alias to a database path.
When defining an alias, the syntax is still unchanged:
base1 = c:\databases\mybase1.fdb
base2 = c:\databases\mybase2.fdb
emp = $(dir_sampleDb)\employee.fdb
...

However, the syntax becomes a bit more complex when you


want to set other parameters for a database. You add the desired
parameters between curly braces {}, immediately below the line,
that sets the alias of the database, for example:
#Example
emp = $(dir_sampleDb)\employee.fdb
{
UserManager = LegacyUserManager
WireCrypt = disabled
}

Appendix
117

That example defines an alias named emp pointing to the sample


database (employee.fdb). It also specifies that the user authentication
plug-in for that database is LegacyUserManager and that wire
encryption is disabled.
Everything after a # is a comment, and ignored.

The parameters below affect the database engine. Their functions


can be seen in the release notes, or in the comments contained in the
configuration files.
DatabaseGrowthIncrement
DeadlockTimeout
DefaultDbCachePages
EventMemSize
FileSystemCacheThreshold
ExternalFileAccess
GCPolicy
LockAcquireSpins
LockHashSlots
LockMemSize
MaxUnflushedWrites
MaxUnflushedWriteTime
SecurityDatabase
ServerMode
UserManager
WireCrypt
WireCryptPlugin

The following parameters are “client side”.


AuthClient
Providers

These parameters can be set only at connection time, using the


DPB/SPB.
ConnectionTimeout
DummyPacketInterval
IpcName
RemoteAuxPort
RemotePipeName
RemoteServiceName
RemoteServicePort
TCPNoNagle

Appendix
118

Glossary
BDE: Borland Database Engine, which allows access to many
databases, including dBase, Paradox, InterBase, SQLServer, Oracle,
etc. Projects developed with the Borland programming languages
such as Delphi, CBuilder, etc. made extensive use of BDE, which
made developer’s life easier, at a cost in performance, because BDE
could not exploit DBMS specific features. BDE is currently
discontinued.

BLOB: The term was not originally an acronym but a reference


to the 1958 Steve McQueen horror movie. Blobs as database objects
appeared first in DEC's Rdb/ELN. Now the term has gained the
retronym Binary Large OBject as opposed to CLOB, a character
based large object. In Firebird, blobs store binary or textual data of
arbitrary size. In Firebird, blobs can have different subtypes, e.g.,
text or binary.

BLR: Again, the term BLR was not an acronym although the
letters happen to be the initials of the manager of the Rdb/ELN
project at DEC. By design, BLR was a single format that could
represent queries in SQL, or the extinct QUEL and GDML
languages. Now it carries the retronym Binary Language
Representation and is the language used inside the Firebird engine.
Procedures, triggers, views, etc. are compiled into BLR, and stored
in the database.

Character set: Determines the characters symbols available for


use. Most languages share some characters, like the digits from 0 to
9. Other characters are specific to a language or a few languages.
Each language has a set of characters that its speakers expect. The
mapping from characters to binary values varies between countries
and computer manufactures and even among applications on a
platform for a single country. Mappings from a set of characters to
binary values are called character sets or code pages. Firebird
supports many character sets. However, if you are writing an

Glossary
119

application that handles more than one language, consider using the
universal encoding UTF-8 even though it uses more space than a
character set designed for a specific language.

Collation: Determines how the characters of a given character set


are ordered relative to each other. Defining a collation, tells Firebird
that an “a” compares and sorts equal to an “ã”, “à”, “â”, etc., or that
"a" precedes and is less than “ã” which precedes and is less than “à”
which precedes and is less than “â”. Different countries use
different collations. Sometimes a single country uses different
collations in different applications. Firebird supports many
collations.

Commit: Close a transaction and make all changes associated


with it durable parts of the database, visible to other transactions

CommitRetaining: Commits a transaction, making all its


changes durable in the database. Unlike the Commit, the
CommitRetaining maintains transaction context. Do not use
CommitRetaining in long-running transactions because it impairs
performance by blocking garbage collection.

Concurrency: See Transaction isolation.

Consistency: See Transaction isolation.

dbExpress: Data access interface to databases created by


Borland. dbExpress enables you to standardize the method of
accessing data, regardless of which DBMS you use, simply by
specifying the appropriate driver.

DBMS: Database Management System. A program that allows


you to store, modify and extract information from a database. The
DBMS receives requests for information from applications and
returns the appropriate data. A DBMS manages concurrent usage
and prevents conflicts between users.

DDL: Data Definition Language – subset of the SQL language


that manipulates metadata (database structure).

Glossary
120

Dirty Read: See Transaction isolation.

Delphi: IDE/Programming language generating executables for


Windows, MacOS, iOS and Android, based on Pascal, created by
Borland and currently owned by Embarcadero©. Widely used in Brazil,
Russia, and several other countries.

DML: Data Manipulation Language – subset of the SQL


language that manipulates user data.

DPB: Database Parameter Block – an in-memory buffer


containing configuration parameters, passed to Firebird through its
API when connecting to or creating a database.

DSQL: See Dynamic SQL .

Dynamic SQL: SQL statements available for programs that


build queries at run time and send them to the DBMS. Delphi is one
language that creates Dynamic SQL statements.

Embedded Server: Firebird configuration that allows you to


access databases without installing Firebird.

FDB: Adopted as the default file extension for databases created


in Firebird 1.5 and later. The file extension is just a convention;
Firebird can open databases with any extension.

FireBase: Biggest portal dedicated to Firebird in Portuguese


language, offering discussion lists, articles, FAQ, support, etc.:
www.firebase.com.br

Firebird Foundation: A foundation created to insure the


continuous development of the Firebird by raising money to pay the
Project’s developers.

Foreign Key: Referential integrity constraint - a foreign key is a


field (or collection of fields) in one table that uniquely identifies a
row of another table.

Glossary
121

Gbak: Firebird command line utility that backs up and restores


databases. When gbak backs up a database, it makes a copy of the
database's metadata and data. When gbak restores a database, it
creates a new empty database and populates it with the copied
metadata and data.

GDB: Default file extension used in InterBase© databases. Refers


to the name of Groton DataBase Systems, old owner of InterBase©.
The gdb extension is not recommended because it has significance
on Windows systems.

Gfix: Firebird command line utility used to verify and correct


some physical database corruptions. It can also set parameters
stored in the database header.

Gsec: Firebird command line utility responsible for managing


users in the Firebird server. Deprecated in Firebird 3.

Gstat: Firebird command line utility that extracts statistics from


the database like number of records in a table, index depth, etc.

IBO: InterBase Objects. Delphi and CBuilder’s native access


components package, developed by Jason Wharton and distributed
under the Trustware license.

IBSurgeon: Russian company that specializes in database


performance optimization and recovery tools (www.ibsurgeon.com).
FireBase is the official IBSurgeon partner in Brazil.

Interactive SQL: Command line utility that connects to a


database, sends queries and formats their results.

InterBase Objects: See IBO.

ISQL: See I nteractive SQL .

Glossary
122

JRD: Jim’s Relational Database. It is the name of the


InterBase©/Firebird’s kernel, in reference of its creator, Jim Starkey.

MGA: Multi-Generational Architecture. Also known as


Versioning or MVCC. Is the name of the concurrency management
system designed by Jim Starkey and used in the InterBase, Firebird
and PostgreSQL databases.

NetBEUI: NetBIOS Extended User Interface (or WNET), is a


NetBIOS-based network protocol, very simple, supporting a
maximum of 255 computers.

ODBC: Open Database Connectivity. A common interface


allowing applications to access various DBMS using a specific
ODBC driver for the desired DBMS.

ODS: See On Disk S tructure.

On Disk Structure: Indicates the version of the database


structure. Firebird 3 supports only ODS 12.

Primary key: Identifies a single record in a table. Can be


composed of one (single key) or more (composite key) fields of this
table. In Firebird, when a primary key is declared, a unique index is
automatically created and associated with it.

Procedural SQL: Language used in Firebird for writing triggers,


procedures and code blocks.

Procedure: See Stored Procedures.

PSQL: See Procedural SQL.

RDBMS: Relational Database Management System. Databases


based on the relational model created by E.F. Codd. Allows the
definition of data structures, operations of storage and retrieval of
information, and definition of integrity rules. The data is arranged in
tables. A table is a collection of rows (or records). Each row in a

Glossary
123

table contains the same fields. Relationships between tables are


based on data in their fields.

ReadCommited: See Transaction isolation.

Referential Integrity: Allows the definition of a master-detail


relationship between tables, enforcing the rule that a detail record
must have an associated master record, unless the detail record can
have null values in the fields that reference the master record.

Repeatable Read: See Transaction isolation.

Repeatable Read Stability: See Transaction isolation.

RollBack: Cancel a transaction, undoing all changes it made. A


rollback ends a transaction.

RollBackRetaining: Cancel a transaction, undoing all changes it


made. Unlike the RollBack, RollBackRetaining maintains the
transactional context. Avoid using RollBackRetaining in long-running
transactions because it inhibits garbage collection and degrades
performance.

SPB: Service Parameter Block – in-memory buffer passed to


Firebird via its API, to set parameters of the Firebird Services API.

SQL: Structured Query Language. The language used to "talk"


with relational databases. As a declarative language, it specifies what
the result you want, and not how the engine should get it.

Stored Procedures: Routines stored inside the database written


in PSQL.

Sweep: The process of reading all the database tables, removing


rolled back records and record versions that can no longer be read
by active transactions, and resetting the "Oldest Interesting
Transaction". Can be started manually or automatically.

Glossary
124

System Tables: Tables used by Firebird to store the definition


of the database metadata. Their names start with RDB$.

Transaction: Logical unit of work comprising one or more


commands/operations performed on the database. If the
transaction fails, all the operations associated with it can be undone
(rolled back), or made permanent (commit).

Transaction Isolation: Determines how a transaction will


interact with other transactions accessing the same database,
regarding the visibility of data. There are four isolation modes. Dirty
Read allows a transaction to read another transaction's results before
they are committed. Read Commited allows a transaction to read any
committed data. RepeatableRead (Concurrency) allows a transaction to
see consistent results, ignoring changes committed during its
lifetime. RepeatableRead Stability (Consistency) guarantees that the
results of the actions of concurrent transactions are the same as if
the transactions ran one after the other in some order. DirtyRead
mode is not supported by Firebird.

Trigger: Routines associated with a table, started when some


action happens in this table, like an insert, delete, etc.

UDF: User Defined Function –Feature that allows users to


create external functions to be used inside the database. UDFs are
stored in shared libraries, DLLs on Windows or Shared Objects on
Linux.

Unique key: value or values that uniquely identify each record in


a table, distinguishing them from other records.

XNET: Transport protocol available only on Windows, allowing


local connections using shared memory.

WNET: See NetBEUI.

Glossary

You might also like