For the Polyhedra-based application to work faster in such environments, there has to be some element of parallelism in the application. If one simply has a single client launching queries and transactions on the database in a serial fashion, the database itself will not go faster - but there will of course be extra CPU cycles available for any other processing that may be going on in the computer. If there a multiple clients using the Polyhedra database, then not only will the individual processes be able to be handled by separate processors, but also the separate threads in the database can be executed in parallel in separate processors.

 

(The database uses the threads for two purposes: for fault-tolerance/persistence support, and for query processing. A separate thread is used for transaction logging and for keeping the standby up to date, to avoid the database being blocked whilst this is being done. For query handling, there can either be a separate thread for each connected client, or the database can be configured to use a pool of threads that are shared between the current client connection. To allow queries to be done in parallel, an internal table-level, multiple-read, single-write locking mechanism is used. The overall effect is that a time-consuming query will not completely block queries being done on behalf of other clients; each query will see a self-consistent view of the data; and, performance is enhanced on SMP systems.)

What programming interfaces are available for the database??

Several APIs exist that can be used to interface external applications to a Polyhedra server:

• Callback Client API - This API contains all of the active query management to interface to any Polyhedra server. This API supports active and static queries for both SQL and direct object queries. It is based on the callback model of programming (hence its name) and so is well-suited to the event-driven nature of Polyhedra.
  (NB - the libraries for this API are only provided for certain editions of Polyhedra.)
  
  
• ODBC API - This API contains all of the active query management to interface a client application to any Polyhedra server. This API supports active and static queries for both SQL and direct object queries. It is based on the standard ODBC programming interface, but with extensions to better support the event-driven nature of Polyhedra. Polyhedra recommend the use of this API rather than the older, Callback API, as it is standards-based; also, if use is made of the SQLPrepare/SQLExecute functions, then it imposes less load on the server. On all platforms, ODBC client applications can be linked with the Polyhedra library directly, allowing them to make use of some Polyhedra-specific extensions to ODBC; alternatively, on Windows and many Linux platforms the Polyhedra library is available as an ODBC driver - in this case, the client applications would be linked with the corresponding ODBC driver manager
  
  
• JDBC API - released in Q4 2000, this allows the Java programmer to access a Polyhedra database; the accomPanying library is written in Java, and so works with any Java engine. It uses the Polyhedra Native Protocol over TCP/IP to communicate with the Polyhedra Database Server. The release kit also contains a second set of class definitions, for use with Java engines such as CDLC that do not support JDBC.
  The driver supports JDBC 2.0 with some limitations. For 'Scrollable' result sets, all standard cursor movement methods are allowed; for 'updateable' result sets, moveToInsertRow(), updateXXX(), insertRow(), updateRow(), deleteRow(), etc., are supported. All the following settings are supported: TYPE_FORWARD_ONLY, TYPE_SCROLL_INSENSITIVE, TYPE_SCROLL_SENSITIVE.
  
  
• ADO.NET and OLE DB - For use by Windows programmers, both an ADO.NET and an OLE DB data provider is available, which allows .NET applications and applications that understand OLE DB to access the Polyhedra database in an efficient fashion; both data provides make good use of the Polyhedra active query mechanism to notify the user code of changes in the database as they occur.

In addition, there are a number of special-purpose APIs that are collectively referred to as the Polyhedra embedded API (and documented in the 'Polyhedra on embedded systems' reference manual). These APIs can be used to 'fine-tune' the functionality of the Polyhedra database server, by providing alternative definitions of certain functions (and thus allow file-system calls to be intercepted, for example) or to provide tuning parameters to the memory management system used within the Polyhedra server.

Finally, for platforms on which Polyhedra provides an ODBC API or for which there is a Java engine that can use our JDBC driver, other options become available. For example, PHP and Python programs can interact with Polyhedra via ODBC. See here for more details.

Does Polyhedra support pessimistic locking?

Yes! With the release of Polyhedra 8.7, we now support both optimistic and pessimistic locking. We recommend the use of optimistic locks rather than pessimistic locks, as the latter are not really suited for real-time applications. 

What arbitration mechanism should I be using?

In a fault-tolerant set-up, it is important to ensure that the two Polyhedra servers know which should be master, and which should be standby - for if both think they are master, they will be operating independently (with no data synchronisation) whereas if both think they are standby, neither will be able to connect to the one they think is the current master. To allow you to control this, and determine when switch-over should occur, the Polyhedra servers use an external arbitration mechanism.

Polyhedra provides three ways for the server to communicate with the arbitration mechanism, and the choice of the one to use will depend on individual circumstances. The three ways are

• Using an arbitration database

  This was the earliest of the three options, and is appropriate in low-volume applications, where it is not worth developing a hardware-based solution to avoid the cost of an extra computer per deployed system. In this scenario, there are two machines each running one of the Polyhedra servers that act as a fault-tolerant pair, while an independent database runs on a third machine. The third machine can be very low-specification, or it could be running other services to the overall system - a print server, say, that does not of itself need to be fault-tolerant.

  the arbitration database would have a table containing a record for each of the two FT servers, and when they start up each of the FT servers would connect to the arbitration database (using a fault-tolerant connection) and attach to 'its' record using an active query. Periodically, each FT server would update a time stamp field in its record, so that CL code running in the arbitration database (or a separate client application running on the arbitration machine, monitoring the arbitration database) can determine whether it is still live. A field in each record says whether the relevant FT server should be master or standby, and another field gives the heartbeat interval telling the server how often it should update the time stamp attribute.

  When one of the FT servers starts up, it needs to be able to connect to the arbitration database, and also - if it is to be the standby - it needs to be able to connect to the master. Apart from that, both the FT servers can survive if they temporarily lose connection to the arbitration database, provided they are in contact with their partner server. When the arbitration database restarts - or connection is re-established if it was a network failure - it allows the FT servers to tell it what mode they are currently operating in, whereas at all other times they take their orders from it. Thus, one has a resilient, software-only mechanism that survives a failure of one of the three machines or servers.

  Sample code illustrating the such a set-up is provided on request to customers and registered evaluators.

• A simple TCP-based protocol

  In embedded applications that are going to be widely deployed, having an extra board or computer to act as an arbitrator would push the unit cost of each system too high. Instead, one would develop a hardware-based system so that one can reliably tell which of a pair of boards should be master, and which the standby. To allow your software to tell a Polyhedra server the master/standby status of the board on which it runs, a simple-TCP-based protocol has been developed. In its simplest form, you would have a process that acted as the server end of this protocol, that accepted a connection from the database process; whenever a message was received from the database server, a reply would be generated containing an indication of the board state (and thus the require master/standby state of the Polyhedra server). An example C-coded application is provided in the Polyhedra release kits. One could design a system where the arbitration processes on the two boards / machines could communicate with each other, and decide between themselves the master/standby staus. However, there are risks involved in such a setup, as it assumes an utterly reliable connection between the boards, such that a loss of a TCP/IP connection can be taken as an infallible indicator that the other server or the other board has failed.

• A simple protocol over LINX (OSE signals)

  This is provided for the same reason as the TCP-based protocol option described above, but uses LINX (also known as OSE messages) as the transport. As a result, it is generally more appropriate to use this rather than the TCP-based option when deploying fault-tolerant Polyhedra systems under OSE, or when deploying Polyhedra alongside Enea Element (which makes heavy use of LINX).

So, to summarise, the RTRDB-based arbitration solution is only appropriate when just a few systems are likely to be deployed, or when each system will already have to have a third machine with some spare capacity. In other circumstances, the solution based on LINX will be the best if running on the OSE RTOS or alongside Element, and the simple TCP/IP-based solution is applicable in all other cases.

Does generation of a snapshot block transactions on the database?

(The following answer applies to Polyhedra32 IMDB, Polyhedra64 IMDB and Polyhedra Lite. Polyhedra Flash DBMS is different, in that it does not do journal logging (except to update a running standby server), and instead uses a technique known as shadow paging to maintain a consistent copy of the database on file.)

If one issues a ‘save’ command when using Polyhedra IMDB, the database server sets a flag on each persistent record to say that it needs to be sent to the new snapshot. Then, using a separate thread, it starts writing the new snapshot to a temporary file – first a copy of the schema and (if CL is being used) a copy of the compiled bytecode, followed by the database contents. When writing out the database contents, it does it in chunks, clearing the flag as it copies each record into a buffer.

While the dump is in process, other database transactions can occur. Queries cause no difficulties, but when it comes to updating or inserting records that still have the flag set, the database server clears the flag – but first puts a copy of the original state to one side, and it is this copy that is sent to the snapshot file when its turn comes.

When the snapshot file is complete, the original file is renamed as a backup, and the temporary file is given the name of the original file. (This avoids a ‘window of vulnerability’ where there is no valid snapshot. To guard against a system crash while the rename games are taking place, a start-up script can check for the existence of the database file before starting the database, and if not present it can rename the backup file as the database file.) The database server now writes new journal records to the end of the new snapshot, including any journal records that have been cached whilst the snapshot is in progress.

If the ‘save into <filename>’ form had been used, then the process is similar, except that the database server does not need to cache journal records: it can write them to the end of the main database file, as normal.

Thus, producing a snapshot does not block the database from servicing queries and other transactions. The only caveat is this: if a ‘save’ command is issued (not a ‘save into’), and another client issues a transaction using our ‘safe commit’ mechanism (where the acknowledgement of the successful transaction is delayed until the server has journalled away the transaction), then the acknowledgement is delayed until the journal record is written to the NEW database file.

What is the effect of journal_queue_size on the above? Well, if the database is being updated very quickly, the limit on the size the journal queue means the database server will start blocking transactions if the journal records are being produced too fast for the disk (or the standby, when one is present) to keep up. With regards to snapshots, it affects how much journal information can be cached before the database server stops blocking transactions that alter the database contents. Note that the journal queue limit is a size constraint, not a limit on the number of transactions – so it is possible that one big transaction might fill up the queue, whereas a number of smaller transactions might only partly fill it.

Turning now to standby start-up, the main database server uses a similar technique – except that it has to send over all (non-local) records, not just the persistent stuff. The master server does not consider the standby being ready until the initial snapshot has been transferred – so if a transaction with the safe-commit flag is performed after the standby has gone ready but before all the cached journal records have been transferred, acknowledgement of the new transaction will have to wait until the standby has fully caught up. As before, the maximum journal queue size might mean that the server has to block transactions – particularly if the database is large, the network is slow, and the transaction rate is high.