Answers to some Frequently Asked Questions
General Questions about the product family
What is Polyhedra?
Polyhedra is a family of relational database management systems
with a large degree of common functionality, including the same SQL engine and the same client-server protocols. Polyhedra is designed for use in embedded systems, so it is small and fast, and can be used in applications where regular administration is impractical or impossible.
What was Polyhedra designed for?
Polyhedra was designed to provide a coherent set of development tools for database applications in several specific market areas normally not well served by conventional database vendors (both Object-oriented and Relational). The core Real Time Relational Database (RTRDB) is designed to sit in an Open Systems environment employing TCP/IP and UDP for connections to clients and other database servers.
- Real Time Data Management - The heart of Polyhedra is a fast memory-resident relational database management system called the rtrdb, that is ideally suited to applications requiring both real-time responses to data changes coupled with the flexibility of modern database technology. Such applications exist in the markets of Process Control and Automation, Telecommunications, Point of Sale, Materials Handling and many others.
- Embedded Systems - As the Polyhedra database is memory resident, it is ideally suited to embedded systems that are often diskless. The Polyhedra database has been successfully employed in several embeded systems running on a variety of target achitectures, including Intel x86 and PowerPC-based systems.
- System Front Ends - The RTRDB is capable of communicating directly and actively with other databases, systems and external devices. This makes it an ideal choice for integrating a lot of disparate data systems together into a single coherent application.
The software contains standards-conforming APIs for those wanting to write client applications in C, C++ or Java, and for use on Windows and Linux/x86 it also has an ODBC driver; on Windows, there is also an OLE DB data Provider. As client applications can be colocated or remote from the database(s) they are using, great flexibility is provided in the organisation of the overall system.
What is different about Polyhedra?
Polyhedra offers significant advantages over traditional DBMS systems:
- Memory Resident - The Database server for Polyhedra IMDB is entirely memory resident. This provides performance that is significantly greater and more predictable than disk-based alternatives. Independent research shows in-memory databases to be an order of magnitude faster than disk based systems, even where the disk based system is cached into RAM.
Data is persisted (written to a non-volatile store) by the creation of a journal file. Data be marked as persistent or transient to control whether changes to it are journalled, and an application can ask to be notified of a transaction's completion either after the transaction has committed in main memory, or after the journal record has been written. The timing of the creation of a "snapshot", a checkpoint of the current state of the database, is under the control of the application.
(The Polyhedra family now also includes Polyhedra FlashLite
, where the master copy of the data resides in a file, and there is a configurable cache to allow the database designer to balance performance and memory consumption.)
- Fault Tolerant - Polyhedra can be configured so that servers run in a hot-standby arrangement, so that if the master database fails the standby can immediately take over in milliseconds or less.
- Active Queries - Polyhedra incorporates a unique concept called active queries, which allow connected clients to monitor the database for changes to selected information, in a timely and scaleable fashion. No polling needed!
- Active Database - Polyhedra supports the creation of scripts which are triggered by data changes. By use of these, the database becomes an active component in any application and not just a passive data repository like many others.
- Heterogeneous Client-server Architecture - the separation of the the server (and the data it holds) from the client application code protects the database from rogue code, especially in environments where memory-protection is enabled. The platform-independence (heterogeneity) of the protocol used means that clients can be running on separate machines, using different operating systems and processor types to that on which the database server is running.
When was Polyhedra first released?
The first release of Polyhedra32 IMDB was in 1993; see the history section
for more information.
What are in the release kits? How do I install Polyhedra?
A polyhedra release kit has a name such as
win32-i386-msvc-08.06.0001.zip, which indicate not only what version of Polyhedra it contains, but also the platform for which it is intended - so here we have an 8.6.1 release kit for Polyhedra32 IMDB, for use on Windows on x86-architecture machines, with libraries suitable for use with the Visual C++ compiler. (By contrast, the file name for the corresponding Polyhedra64 IMDB release kit for use on 64-bit Windows platforms has the name
win64-x86_64-msvc-08.06.0001.zip, and the Linux/PowerPC release kit is called
linux-powerpc-gcc-08.06.0001.tar.gz.) The version naming scheme follows a standard major.minor.bugfix convention, so 8.0.0 was the first major release after the Polyhedra 7 series, and Polyhedra 8.4.1 was a bug-fix release for Polyhedra 8.4. Typically, a major release indicates a significant change in the functionality necessitating a substantive change in the protocols, file formats or APIs, whereas for a minor release the changes are minimal if you do not use the new functionality introduced by the new release. In all cases, though, the Polyhedra compatibility principles mean that adopting the new release is straightforward, and field-upgrades can almost always be done with zero downtime (if using a fault-tolerant server configuration).
In the case of Windows, the release kit is a ZIP file, whereas for other platforms it will be a .tar.gz file; in each case it contains a compressed directory image, and in essence all that needs to be done is to uncompress it. Thus on Windows, there is no need to alter the Windows registry; assuming a Polyhedra 8.7 release kit for Windows has been unpacked into, say, the C:\poly8.7 you would simply ensure that the PATH environment variable included the directory C:\poly8.7\win32\i386\bin. (This is covered in more detail in the installation guide document, included in the release kits.) The contents of the Polyhedra 8.7 release kits for Windows/x86 have the following structure:
a guide to getting started with Polyhedra. The latest version of this manual is available for download here.
a document summarising the features that differ (in their availability) between different editions of Polyhedra.
how to install Polyhedra. The latest version of this manual is available for download here.
Release notes for this release of Polyhedra, including a brief overview of newly added features, and information about migrating applications to use the new release.
a text file containing the version number of the new release.
a directory containing example software. The 'clients' subdirectory contains the source code for several
applications that are also supplied in executable form in the release
kits; these example applications are described in the evaluation guide,
and used in the worked exercises in that document (which you are
strongly advised to also follow for yourself, if you are new to
Polyhedra). Demo_1 to demo_4 are used by the worked exercises in the
evaluation guide, and the other subdirectories contain code illustrating
particular Polyhedra features - read the text file in the subdirectory
header files for use when compiling your own client applications.
While all the other directories will have exactly the same contents
for all the release kits for a particular version of Polyhedra, this
directory is specific to the Windows release kits. Release kits for other platforms will have a similar, appropriately named directory with a similar structure: that is, a doc and examples directory, and a subdirectory named after the hardware architecture with a copy of the binary executables and (in a subdirectory named after the compiler toolset) some object modules and libraries suitable for use with the indicated compiler.
This contains text files describing how to compile and link client applications on this platform. It also contains instructions on relinking the main Polyhedra executables, to allow you to supply alternative definitions of certain functions - note that for certain editions of Polyhedra this will not be possible as the release kits omit the libraries you would need to achieve this.
some platform-specific examples.
the main Polyhedra executables plus some client applications in executable form,
the libraries you will need to build and run your own client applications. For some editions of Polyhedra, this subdirectory will also include the libraries needed to relink the main Polyhedra executables.
In addition to the main release kits, there are also some supplementary, platform-specific release kits. Thus, for Windows and Linux/x86 we supply an ODBC driver kit that allows you to install the standard Polyhedra ODBC library as a driver for use with an ODBC driver manager. The Polyhedra JDBC driver is also supplied as a supplementary release kit; as it is a type-4 driver written purely in Java, it is platform-independent. The reference manuals are not included in the release kits, but can be obtained individually from here
; customers with a support contract can also download the complete reference manual kit as a single ZIP file.
Does Polyhedra support the relational model?
Yes, Polyhedra is relational
. It also has some object-oriented features (such as table inheritance
), but as far as client applications are concerned, it can be thought of a relational database system that allows queries and updates via its SQL engine.
What data types are supported?
Polyhedra supports 64-bit, 32-bit, 16-bit and 8-bit signed integers; 64-bit and 32-bit floating point values; variable-length character strings (both capped-length and unbounded); unbounded-length byte sequences; booleans; and, datetime (with microsecond resolution).
Does Polyhedra support entity and referential integrity?
In a word, yes.
Entity integrity says that record must have the right structure: for example, is a table all records have the same number of columns/attributes, and the value placed in a column is of the right type. Thus, you cannot store a string value or a floating-point value in a cell if the column is flagged as having an integer type. Polyhedra enforces this strictly - though if you put a 64-bit floating value into a cell flagged as holding a 32-bit quantity, for example, some accuracy is lost. Polyhedra also enforces 'not null' and uniqueness contraints - thus it ensures the values in the primary key attributes are check to be not not null, and the value in the primary key column (or the combination of values, if the table has a multi-column primary key) uniquely identify that record. Any operation that breaks entity integrity automatically causes the transaction to be aborted, with a suitable error response to the client that initiated the transaction.
Referential integrity says that if a column (or combination of columns) is defined to be a foreign key to a record in another table - or even a record in the same table - then for each row the value(s) in the cell(s) making up the foreign key are either null, or identify a single record in the referenced table. Polyhedra allows foreign keys to be defined, and aborts all transactions that break referential integrity. (However, Polyhedra does not perform cascaded deletes, where the deletion of a record will cause the deletion of all records that reference it.)
Does Polyhedra extend the relational model?
Yes. Firstly, Polyhedra allows the database designer to define a table as being 'derived from' another. So a table y that is derived from table x inherits all the attributes/columns that x has (including the primary key attributes), and can have additional columns. And of course table x can derive from y (inheriting all its columns, including the columns inherited from x) and add further columns. When table t is added, deriving from y say, it can have additional columns with the same names as columns added in table z, and there are no problems with clashes. From the point of view of the database user, tables x, y, z and t all appear as standard tables, and can be populated, updated and queried via SQL in exactly the same way as normal; no special syntax is required. The main advantage of table inheritance is that it allows a more accurate representation of the real world. Thus, in a process control environment, a digital sensor and an analogue sensor have many things in common but their values are of different types, and an analogue sensor would normally have conversion factors (to convert from raw measurements to engineering limits) and alarm limits; all of this is easy to model with a base table called sensor and derived tables called digital and analogue.
A second extension from the relational model is to allow columns/attributes to be flagged as 'shared' or 'virtual'. A shared column is one where all records in the table (and derived tables) share the same value. Update the shared value in one record, and the new value appears in all records. The shared value does not take up space in the record; instead, it is stored with the definition of the table in the schema. A virtual attribute is similar - except that derived tables keep their own copy of the value, at each level. Thus, if the sensor table has a virtual attribute called tabletype, it is easy to ensure all analogue sensors show the value ANALOGUE (or ANALOG if you prefer American English!) and digital sensors show the value DIGITAL.
What are active queries?
Polyhedra's active query mechanism provides a way for a client application to monitor record(s) in a Polyhedra database without having to poll. Basically, the client application launches a query almost in the normal way - but instead of the server forgetting about the query once it has been answered, it remembers enough about the query so that it can update the client when the records are changed. This is covered in more detail here.
What is the historian module?
Polyhedra 32 IMDB and Polyhedra 64 IMDB have a historian module that can be configured (by creating records in special control tables in the database) to monitor selected tables, and to capture time-stamped samples either periodically or when the values change. The time-series data is kept in a file that is treated as a large circular buffer, and be queried via SQL. Old samples can be archived before they overwritten, and archive files can be brought back online so that they can be queried. This is covered in more detail here
Can Polyhedra take advantage of SMP and hyperthreaded computers?
As is common in computing, the answer is a conditional yes!
The Polyhedra database server uses multiple threads to do its work, and so - assuming of course that the operating system has been configured to allow the standard pthreads library to take advantage of the hardware feature, and depending on what the database has been asked to do - it should run faster on SMP machines and machines with hyperthreaded processors. As the Polyhedra software runs as an unprivileged user process, it does not need to be compiled specially for this to work.
For the Polyhedra-based application to work faster in such environments, there has to be some element of parallelism in the application. If one simply has a single client launching queries and transactions on the database in a serial fashion, the database itself will not go faster - but there will of course be extra CPU cycles available for any other processing that may be going on in the computer. If there a multiple clients using the Polyhedra database, then not only will the individual processes be able to be handled by separate processors, but also the separate threads in the database can be executed in parallel in separate processors.
(The database uses the threads for two purposes: for fault-tolerance/persistence support, and for query processing. A separate thread is used for transaction logging and for keeping the standby up to date, to avoid the database being blocked whilst this is being done. For query handling, there can either be a separate thread for each connected client, or the database can be configured to use a pool of threads that are shared between the current client connection. To allow queries to be done in parallel, an internal table-level, multiple-read, single-write locking mechanism is used. The overall effect is that a time-consuming query will not completely block queries being done on behalf of other clients; each query will see a self-consistent view of the data; and, performance is enhanced on SMP systems.)
What programming interfaces are available for the database??
Several APIs exist that can be used to interface external
applications to a Polyhedra server:
- Callback Client API - This API
contains all of the active query management to interface to any
Polyhedra server. This API supports active and static
queries for both SQL and direct object queries. It is based on
the callback model of programming (hence its name) and so is well-suited
to the event-driven nature of Polyhedra.
(NB - the libraries for this API are only provided for certain editions of Polyhedra.)
- ODBC API -
This API contains all of the active query management to interface a
client application to any Polyhedra server. This API supports active and
static queries for both SQL and direct object queries. It is based on
the standard ODBC programming interface, but with extensions to better
support the event-driven nature of Polyhedra. Polyhedra recommend the
use of this API rather than the older, Callback API, as it is
standards-based; also, if use is made of the
functions, then it imposes less load on the server. On all platforms,
ODBC client applications can be linked with the Polyhedra library
directly, allowing them to make use of some Polyhedra-specific
extensions to ODBC; alternatively, on Windows and many Linux platforms the
Polyhedra library is available as an ODBC driver - in this case, the
client applications would be linked with the corresponding ODBC driver
- JDBC API -
released in Q4 2000,
this allows the Java programmer to access a Polyhedra database; the
accomPanying library is written in Java, and so works with
any Java engine. It uses the Polyhedra Native Protocol over TCP/IP to communicate with
the Polyhedra Database Server.
The release kit also contains a second set of class
definitions, for use with Java engines such as CDLC that do not support
The driver supports JDBC 2.0 with some limitations.
For 'Scrollable' result sets, all standard cursor movement methods are allowed; for 'updateable' result sets,
deleteRow(), etc., are supported. All the
following settings are supported:
- ADO.NET and OLE DB -
For use by Windows programmers, both an ADO.NET and an OLE DB data provider is available,
which allows .NET applications and applications that understand OLE DB to
access the Polyhedra database in an efficient fashion; both data provides make good use
of the Polyhedra active query mechanism to notify the user code of
changes in the database as they occur.
In addition, there are a number of special-purpose APIs that are
collectively referred to as the Polyhedra embedded API (and documented
in the 'Polyhedra on embedded systems' reference manual). These APIs can
be used to 'fine-tune' the functionality of the Polyhedra database
server, by providing alternative definitions of certain functions (and
thus allow file-system calls to be intercepted, for example) or to
provide tuning parameters to the memory management system used within
the Polyhedra server.
Finally, for platforms on which Polyhedra provides an ODBC API or for which there is a Java engine that can use our JDBC driver, other options become available. For example, PHP and Python programs can interact with Polyhedra via ODBC. See here
for more details.
Does Polyhedra support pessimistic locking?
Yes! With the release of Polyhedra 8.7, we now support both optimistic and pessimistic locking. We recommend the use of optimistic locks rather than pessimistic locks, as the latter are not really suited for real-time applications.
What arbitration mechanism should I be using?
fault-tolerant set-up, it is important to ensure that the two Polyhedra
servers know which should be master, and which should be standby - for
if both think they are master, they will be operating independently
(with no data synchronisation) whereas if both think they are standby,
neither will be able to connect to the one they think is the current
master. To allow you to control this, and determine when switch-over
should occur, the Polyhedra servers use an external arbitration
Polyhedra provides three ways for the server to communicate with
the arbitration mechanism, and the choice of the one to use will depend
on individual circumstances. The three ways are
- Using an arbitration database
This was the earliest of the three options, and is appropriate in
low-volume applications, where it is not worth developing a
hardware-based solution to avoid the cost of an extra computer per
deployed system. In this scenario, there are two machines each running
one of the Polyhedra servers that act as a fault-tolerant pair, while an
independent database runs on a third machine. The third machine can be
very low-specification, or it could be running other services to the
overall system - a print server, say, that does not of itself need to be
the arbitration database would have a table containing a record
for each of the two FT servers, and when they start up each of the FT
servers would connect to the arbitration database (using a
fault-tolerant connection) and attach to 'its' record using an active
query. Periodically, each FT server would update a time stamp field in
its record, so that CL code running in the arbitration database (or a
separate client application running on the arbitration machine,
monitoring the arbitration database) can determine whether it is still
live. A field in each record says whether the relevant FT server should
be master or standby, and another field gives the heartbeat interval
telling the server how often it should update the time stamp attribute.
When one of the FT servers starts up, it needs to be able to
connect to the arbitration database, and also - if it is to be the
standby - it needs to be able to connect to the master. Apart from that,
both the FT servers can survive if they temporarily lose connection to
the arbitration database, provided they are in contact with their
partner server. When the arbitration database restarts - or connection
is re-established if it was a network failure - it allows the FT servers
to tell it what mode they are currently operating in, whereas at all
other times they take their orders from it. Thus, one has a resilient,
software-only mechanism that survives a failure of one of the three
machines or servers.
Sample code illustrating the such a set-up is provided on request to customers and registered evaluators.
- A simple TCP-based protocol
In embedded applications that are going to be widely deployed, having
an extra board or computer to act as an arbitrator would push the unit
cost of each system too high. Instead, one would develop a
hardware-based system so that one can reliably tell which of a pair of
boards should be master, and which the standby. To allow your software
to tell a Polyhedra server the master/standby status of the board on
which it runs, a simple-TCP-based protocol has been developed. In its
simplest form, you would have a process that acted as the server end of
this protocol, that accepted a connection from the database process;
whenever a message was received from the database server, a reply would
be generated containing an indication of the board state (and thus the
require master/standby state of the Polyhedra server). An example
C-coded application is provided in the Polyhedra release kits.
One could design a system where the arbitration processes on the two
boards / machines could communicate with each other, and decide between
themselves the master/standby staus. However, there are risks involved
in such a setup, as it assumes an utterly reliable connection between
the boards, such that a loss of a TCP/IP connection can be taken as an
infallible indicator that the other server or the other board has
- A simple protocol over LINX (OSE signals)
This is provided for the same reason as the TCP-based protocol option
described above, but uses LINX (also known as OSE messages)
as the transport. As a result, it is generally more appropriate to use
this rather than the TCP-based option when deploying fault-tolerant
Polyhedra systems under OSE, or when deploying Polyhedra alongside Enea Element (which makes heavy use of LINX).
So, to summarise, the RTRDB-based arbitration solution is only appropriate when
just a few systems are likely to be deployed, or when each system will
already have to have a third machine with some spare capacity. In other
circumstances, the solution based on LINX will be the best if
running on the OSE RTOS or alongside Element, and the simple TCP/IP-based solution is
applicable in all other cases.
Does generation of a snapshot block transactions on the database?
(The following answer applies to Polyhedra32 IMDB, Polyhedra64 IMDB and Polyhedra Lite. Polyhedra Flash DBMS is different, in that it does not do journal logging (except to update a running standby server), and instead uses a technique known as shadow paging to maintain a consistent copy of the database on file.)
If one issues a ‘
save’ command when using Polyhedra IMDB, the database server sets a flag on each
persistent record to say that it needs to be sent to the new snapshot.
Then, using a separate thread, it starts writing the new snapshot to a
temporary file – first a copy of the schema and (if CL is being used) a
copy of the compiled bytecode, followed by the database contents. When
writing out the database contents, it does it in chunks, clearing the
flag as it copies each record into a buffer.
While the dump is in
process, other database transactions can occur. Queries cause no
difficulties, but when it comes to updating or inserting records that
still have the flag set, the database server clears the flag – but first
puts a copy of the original state to one side, and it is this copy that
is sent to the snapshot file when its turn comes.
snapshot file is complete, the original file is renamed as a backup, and
the temporary file is given the name of the original file. (This avoids
a ‘window of vulnerability’ where there is no valid snapshot. To guard
against a system crash while the rename games are taking place, a
start-up script can check for the existence of the database file before
starting the database, and if not present it can rename the backup file
as the database file.) The database server now writes new journal
records to the end of the new snapshot, including any journal records
that have been cached whilst the snapshot is in progress.
save into <filename>’ form had been used, then the process is
similar, except that the database server does not need to cache journal
records: it can write them to the end of the main database file, as
Thus, producing a snapshot does not block the database
from servicing queries and other transactions. The only caveat is this:
if a ‘save’ command is issued (not a ‘save into’), and another client
issues a transaction using our ‘safe commit’ mechanism (where the
acknowledgement of the successful transaction is delayed until the
server has journalled away the transaction), then the acknowledgement is
delayed until the journal record is written to the NEW database file.
is the effect of journal_queue_size on the above? Well, if the database
is being updated very quickly, the limit on the size the journal queue
means the database server will start blocking transactions if the
journal records are being produced too fast for the disk (or the
standby, when one is present) to keep up. With regards to snapshots, it
affects how much journal information can be cached before the database
server stops blocking transactions that alter the database contents.
Note that the journal queue limit is a size constraint, not a limit on
the number of transactions – so it is possible that one big transaction
might fill up the queue, whereas a number of smaller transactions might
only partly fill it.
Turning now to standby start-up, the main
database server uses a similar technique – except that it has to send
over all (non-local) records, not just the persistent stuff. The master
server does not consider the standby being ready until the initial
snapshot has been transferred – so if a transaction with the safe-commit
flag is performed after the standby has gone ready but before all the
cached journal records have been transferred, acknowledgement of the new
transaction will have to wait until the standby has fully caught up. As
before, the maximum journal queue size might mean that the server has
to block transactions – particularly if the database is large, the
network is slow, and the transaction rate is high.
How do I...
There is a separate section of this site with some detailed description on how to achieve certain effects with Polyhedra. The most common 'how' question, though, is:
How can I design my application to get the best performance?
There is no hard and fast design rules for this. However, there are some general principles that can be applied:
- Get the schema right
- You have to know your data and how it will be used, and then design the schema accordingly.
- Use inheritance where appropriate - this makes many queries much more efficient, avoiding the need for joins.
- Denormalise to make the 'usual' queries easier (and use CL to maintain the integrity otherwise lost by the denormalisation)
- Add indexes (sparingly) to speed up your normal queries.
- Use varchar(n) rather than char when fields have a known size limit. This way, the field value is stored 'in record' instead of having a pointer to a dynamically-created buffer of the right size.
- make use of CL in the database. This is efficient, with the following benefits:
- it reduces the work of the clients making changes: integrity checks and 'knock-on' changes become the responsibility of the database designer, and client code is simplified;
- integrity checks can be as complex as needed, and can be improved/extended without having to alter client applications; and,
- it run within database, within transaction, and so cheap to perform and reduces client-server traffic.
- Use active queries where appropriate. This avoids polling, less load. However, you should…
- consider putting 'minimal intervals' on your queries; this reduces strain on server and comms when data is changing very rapidly;
- simplify and restrict your queries - in particular, avoid joins,
order by, and the use of
select * (instead, specify the columns you want);
- ensure all shared and virtual attributes are 'hidden'; and,
- consider putting a row limit on your query.
- Reduce client-server interaction. For example, you could...
- batch up changes into larger transactions, where the logic of your application makes this legal and reasonable;
- set off queries, etc in parallel - the library code will combine messages where possible;
- use the 'update and SELECT' mechanism where appropriate. Polyhedra allows a query (other than an active query) to consist of a sequence of semicolon-sepated SQL DML statements (
send) followed by a single
SELECT statement. The changes are done as a single transaction, and, if successful the database then executes the
select statement without any intervening transactions.
- use Polyhedra's
INSERT OR UPDATE SQL extension where appropriate. This can save an application having to check whether a record exists before updating it!
If using ODBC and find yourself frequently using the same SQL statement, differing only in the values supplied, consider using
SQLPrepare once at the start of the program (with question-marks for the values), and
SQLExecute whenever you need to invoke the statement. The values are read from locations you specify and transferred across in native format, so the message size is reduced - and the as the database server will have parsed the statement in advance, it also has less work to do.
- Get the algorithms right!
An obvious point, perhaps, but one that unfortunately needs stressing. Careless coding or database design can cause an operation that should be, say, proportional to the number of records in a table to degenerate to something that is proportional to the square of the number of records, with disastrous consequences to overall performance as the table grows in size!
Users with a support contract can ask the help desk for advice on how to use particular features of the product, and Enea offers consultancy services such as design and code review to help you make best use of the system, and so help you get your Polyhedra-based product working (and to market) faster.