CHAPTER 1. Administration Concepts
This chapter describes the concepts necessary for understanding system administration for ConText Option.
The following concepts are discussed in this chapter:
Administrator Responsibilities
Administrative responsibilities for ConText servers can be divided into two functional areas:
- database administrator (DBA)
Database Administrator (DBA)
The DBA's responsibilities include:
- creating and maintaining Oracle users for ConText Option
- managing the ConText data dictionary
System Administrator
The system administrator's responsibilities include:
- starting and shutting down ConText servers
- monitoring the ConText server queues
ConText Roles
There are three roles provided with ConText Option:
Each ConText Option user should be assigned a ConText role.
For more information about assigning ConText roles to users, see "Managing USers" in "Administering ConText Option (Chapter 2)."
For more information about database roles, see Oracle7 Server SQL Reference.
CTXUSER
The CTXUSER role provides users with the ability to perform text queries. It should be assigned to all users of a text application.
CTXAPP
The CTXAPP role includes all of the CTXUSER privileges as well as the following privileges:
- creating/dropping preferences
- creating/dropping policies
- creating/dropping text and theme indexes
- generating linguistic output
CTXAPP should be assigned to all users who are developing text applications.
CTXADMIN
The CTXADMIN role includes all of the CTXAPP privileges as well as the following privileges:
CTXADMIN should be assigned to users who perform system and database administration for ConText Option.
Note: In general, the CTXADMIN role is only assigned to the CTXSYS user, which is done automatically by ConText Option when the CTXSYS user is created.
Predefined ConText Option Users
ConText Option provides two Oracle users which are created during installation:
CTXSYS
CTXSYS is used primarily to perform administration tasks, such as starting ConText servers and administering the ConText Option queues.
Note: Only the CTXSYS user can start ConText servers.
CTXSYS has the following functions and privileges:
- Oracle DBA with all privileges
- owner of the ConText data dictionary
CTXDEMO
CTXDEMO is used primarily to run the sample applications provided with ConText Option.
CTXDEMO has the following functions and privileges:
- owner of the ConText Option database schema (tables, views, etc.) and data dictionary objects (policies, preferences) used in the sample applications
The sample applications, provided to help application developers develop applications using ConText Option, include:
- EMP and DEPT tables with text indexes for performing ad-hoc text queries
- SQL scripts for setting up text in a ConText Option system for performing text queries
- SQL scripts for performing different types of text queries
- Forms-based application for performing text queries
- SQL script for generating linguistic output necessary for performing linguistic queries and advanced document viewing
- SQL scripts for performing linguistic (theme) queries and viewing Gists
For more information about using the sample applications, see Oracle ConText Option Application Developer's Guide.
For more information about installing and setting up the sample applications, see the Oracle7 Server installation documentation specific to your operating system.
ConText Servers
ConText servers are shared servers that handle text operations in SQL requests. ConText Option server processes mirror their shared Oracle server counterparts, but process only text-related operations.
ConText servers can only be started from the command-line of the server machine where ConText Option is installed. In addition, only the CTXSYS user can start ConText servers.
If the server machine can support multiple ConText servers, multiple ConText servers can run at the same time to help balance the processing load. In addition, ConText servers can work with databases installed on either the server machine or on remote machines.
Text Operations
ConText servers can process five types of text operations:
- DDL operations (creating, dropping, and updating text indexes)
- DML operations (updating the text index for a text column when inserts, updates, and deletes are performed on the table)
- Linguistic Services (generating linguistic output for documents)
The type of text operations processed by each ConText server is determined by the personality mask assigned to the server.
Text Request Queue
Requests for text operations are stored in the Text Request Queue. Available ConText servers monitor the queue for text requests. As text operations are submitted, available ConText servers with the appropriate personalities pick up the operations and process them.
Server Log
ConText Option provides a timestamped report for each action performed by a ConText server. These reports are written to the standard output on which the server was started; however, the reports can be directed to a log file that is specified when the ConText server is started.
For more information about starting ConText servers, see "ctxsrv/ctxsrvx Executables" in "Executables and Utilities (Chapter 8)."
Personality Masks
The personality mask for a ConText server indicates the text operations that the server can process. When a ConText server is started, it is assigned a personality mask. A personality mask consists of one or more of the following five personalities (corresponding to the five text operations supported by ConText Option):
Note: For ConText Option to process text queries, at least one ConText server with the Query personality must be running.
In addition to the user-specified personalities, all ConText servers automatically take on a DBA personality.
The DBA personality does not appear as part of the personality mask because it is automatically assigned to all ConText servers.
For more information about the text operations supported by ConText Option, see "Text Operations" in "Text Concepts (Chapter 3)."
Loader (R)
The Loader personality enables a ConText server to scan directories at regular intervals for files to be loaded into columns in the database. When the ConText server detects a new file in a specified directory, it uses the ctxload utility to load the contents of the file into a specified column.
The directories scanned by ConText servers running with the Loader personality, as well as the scanning intervals and the columns to which the text is loaded, are specified by a ConText object, called a source, which can be attached to a column.
For more information about sources and using ConText servers with Loader personalities for text loading, see "Text Loading" in "Text Concepts (Chapter 3)."
DDL (D)
The DDL personality enables a ConText server to process requests for creating, optimizing, and dropping text indexes. The text index on a column is what allows users to query the text stored in the column.
The DDL personality also enables a ConText server to process DML requests when DML operations are processed in batch mode.
For more information about batch DML processing, see "Text Operations."
DML (M)
The DML personality enables a ConText server to automatically update the text index for a table when changes to the table are made that require the text index to be update. Such changes include inserting new documents, updating existing documents, and deleting existing documents.
Suggestion: When systems have a high volume of text inserts, updates, and deletes, assign the DML personality to multiple servers to better distribute the system load.
Query (Q)
The Query personality enables a ConText server to process queries for text stored either internally or externally in a text column in a database table. If no running ConText servers have the Query personality, queries submitted to ConText Option will fail.
Note: The Query personality is not required to perform queries for the output generated by the Linguistic Services. Linguistic output is stored as structured data and, as such, ConText servers are not required to process queries for this type of information.
Linguistic (L)
The Linguistic personality allows a ConText server to process requests for the Linguistic Services and generate linguistic output. Linguistic output includes:
Note: The Linguistic personality is only required to process requests for Linguistic Services. Once the requests have been processed, Linguistic servers can be shut down or the Linguistic personality can be removed from the personality mask of a running ConText server.
For more information about the Linguistic Services and the output generated by the Linguistic Services, see "Linguistic Concepts" in Oracle ConText Option Application Developer's Guide.
DBA
The DBA personality allows a ConText server to detect and correct client/server failures and perform system cleanup (recovery).
The DBA personality is automatically assigned to each ConText server during start up, which prevents a single point of failure in a multi-server configuration.
ConText Server Monitoring
When a working ConText server detects a failed ConText server, it performs the following DBA actions:
- the working server cleans up the failed server's mailbox and logs the event
- if the failed server was processing a Query request, the working server releases any allocated resources
- if the failed server was processing a DDL request, the working server drops any tables that are not needed for recovery of the requested action
- if the failed server was processing a DML request, the working server places any uncompleted DML requests back in the queue
- if the failed server was processing a Query or DDL request, the working server notifies the waiting client
System Recovery
When a table has a text column with an existing ConText index and the table is dropped without first dropping the index and policy for the column, the index tables and policy for the column remain in the system until recovery is performed.
System recover is performed automatically by ConText servers approximately every fifteen minutes.
System cleanup can also be performed manually using the RECOVER stored procedure in the CTX_ADM PL/SQL package.
Text Request Queue
The Text Request Queue is the logical mechanism ConText servers use to process all text operations, except for text loading.
When a client submits a request for a text operation, such as a query, the request is stored in the Text Request Queue. All available ConText servers scan the Text Request Queue regularly, retrieve pending requests if they have the appropriate personality mask, and perform the requested operations.
The Text Request Queue is actually made up of a number of database pipes and tables:
- pipes for storing Query and DDL requests
Figure 1 - 1 illustrates how ConText servers with different personality masks access the Text Request Queue.
Figure 1 - 1. Interaction Between the Text Request Queue and ConText Servers
ConText servers process text requests in the Text Request Queue in the following order:
Administration --> Query --> DDL --> DML --> Services
When a ConText server finishes processing a request of any type, it always checks the pipes and queues for the next pending request according to the precedence order.
Administration Pipes
Each ConText server has a dedicated administration pipe for administrative tasks that control its operation (e.g. server shutdown).
Query Pipe
When a SQL statement or PL/SQL block includes a text query, the system dispatches the query to the Query pipe.
The ConText servers notify the calling client when the operation is finished.
DDL Pipe
ConText Option dispatches all DDL requests (e.g. CREATE_INDEX, DROP_INDEX, and OPTIMIZE_INDEX) to the DDL pipe.
The ConText servers notify the calling client when the operation is finished.
If a ConText server encounters a problem with a request in the DDL pipe, the error does not affect the pipe or the server processing the pipe. The errored request is recorded as a row in the Services Queue and the server continues processing the remaining requests in the pipe.
The CTX_INDEX_ERRORS view can be used to display errored DDL requests.
DML Queue
The ConText Option DML Queue notifies ConText servers of changes made to a table containing a text column with an existing index.
When a DML operation is performed (i.e. data in a text table is modified, deleted, or added), a request for an index update is automatically recorded in the DML Queue.
Requests are placed in the DML Queue by database triggers that are created automatically the first time a ConText index is generated by a ConText server for the text column.
If DML processing is running in immediate mode, the next available ConText server with the DML (M) personality picks up the requests and updates the index as soon as resources and system load allow.
If DML processing is running in batch mode, the requests are stored in the queue until a user explicitly requests DML processing. Then, available ConText servers with the DDL (D) personality pick up all the pending requests and process the requests as one or more batches.
The DML Queue is persistent. If the database goes down, the contents of the queue remains intact.
The DML Queue consists of two internal tables:
- pending table (DRQ_PENDING)
- in-progress table (DRQ_INPROG)
Note: DRQ_PENDING and DRQ_INPROG are internal tables and should not be accessed directly. To view the queue, use the administration tool or the DML Queue views provided with ConText Option. To administer the queue, use the CTX_DML package.
This table contains one row for each DML operation (request for index update) that has not yet been picked up by a ConText server. The row indicates the specific cell that has changed in the text table.
Note: Requests are visible in the pending table only after the insert, update, or delete has been committed.
When a request has been picked up by a ConText server with the DML or DDL personality, the row is is deleted from the pending table and the server begins the index update. In addition, a row is written to the in-progress table to indicate that the request is being processed.
This table contains one row for each DML request being processed by a ConText server.
Once the ConText server finishes processing the request, the row is deleted from the in-progress table, indicating that the appropriate index has been updated to reflect the document modification that generated the request.
Time-stamps
Requests in the DML Queue are processed in the order they are received, based on a time-stamp. Rows are inserted into the pending table without a time-stamp. At a specified time interval, all unmarked rows within the pending table are marked with a time-stamp. The time-stamp is based on the time each change was committed.
Error Handling
If a ConText server encounters a problem with a request in the DML Queue, the error does not affect the queue or the server processing the queue. The errored request is recorded as a row in the Services Queue and the server continues processing the remaining requests in the queue.
The CTX_INDEX_ERRORS view of the Services Queue can be used to display errored DML requests.
Queue Management
The DML Queue can be enabled/disabled manually to control processing of DML requests. When the DML Queue is disabled, the queue continues to accept requests; however, new requests and any pending requests in the disabled DML Queue are not picked up by ConText servers until the queue is enabled manually.
Services Queue
The Services Queue is used for processing requests for all ConText Option services. The Services Queue is designed to be extensible. As additional services are provided by ConText Option, the Services Queue is the mechanism by which the services will be managed. Currently, the Services Queue supports the following services:
- Linguistic Services (theme and Gist generation)
- error handling for index creation and optimization (DDL) and index updating (DML)
When a request is submitted for Linguistic Services, the request is stored in the Services Queue. A request is picked up by the first available ConText server with the Linguistic personality and the server generates linguistic output for the specified request.
ConText servers with the Linguistic personality pick requests out of the queue based on the request priority and creation time-stamp. Clients may queue a request and continue to work while the request is being processed.
The Services Queue is asynchronous. Clients that place a request in the queue do not automatically block their subsequent requests while waiting for a reply. However, clients can choose to block their subsequent requests for a specified time when they submit requests.
The Services Queue is persistent. If the database crashes after a request has been placed in the queue, the request is still in the queue when the database is recovered.
Services Queue Table (CTX_SVCQ)
The Services Queue consists of the CTX_SVCQ internal table. This table stores a row for each request for a ConText Linguistic Service, as well as the status of the request.
Note: CTX_SVQ is an internal table and should not be accessed directly. To view the queue, you can use the views or administration tool provided with ConText Option.
To administer the queue (e.g. cleaning up entries), you must use the CTX_SVC package or the administration tool.
Error Handling
If a ConText server encounters a problem with a request in the Services Queue, the error does not affect the queue or the server processing the queue. The errored request is recorded as a row in the Services Queue and the server continues processing the remaining requests in the queue.
The CTX_LING_ERRORS view of the Services Queue can be used to display errored requests for Linguistic Services.
Queue Management
The Services Queue can be enabled/disabled manually to control processing of Linguistic Services requests. When the Services Queue is disabled, requests are still accepted into the queue; however, new requests and any pending requests in the disabled Services Queue are not picked up by ConText servers until the queue is enabled manually.
