Chapter 23: Advanced database management
最后更新于：2022-04-01 04:48:25
This chapter provides additional information on each of the supported relational databases in Django, as well as notes and tips and tricks for connecting to legacy databases.
[TOC=3]
## General notes
Django attempts to support as many features as possible on all database backends. However, not all database backends are alike, and the Django developers had to make design decisions on which features to support and which assumptions could be made safely.
This file describes some of the features that might be relevant to Django usage. Of course, it is not intended as a replacement for server-specific documentation or reference manuals.
### Persistent connections
Persistent connections avoid the overhead of re-establishing a connection to the database in each request. Theyre controlled by the `CONN_MAX_AGE` parameter which defines the maximum lifetime of a connection. It can be set independently for each database.
The default value is `0`, preserving the historical behavior of closing the database connection at the end of each request. To enable persistent connections, set `CONN_MAX_AGE` to a positive number of seconds. For unlimited persistent connections, set it to `None`.
#### CONNECTION MANAGEMENT
Django opens a connection to the database when it first makes a database query. It keeps this connection open and reuses it in subsequent requests. Django closes the connection once it exceeds the maximum age defined by `CONN_MAX_AGE` or when it isnt usable any longer.
In detail, Django automatically opens a connection to the database whenever it needs one and doesnt have one already either because this is the first connection, or because the previous connection was closed.
At the beginning of each request, Django closes the connection if it has reached its maximum age. If your database terminates idle connections after some time, you should set `CONN_MAX_AGE` to a lower value, so that Django doesnt attempt to use a connection that has been terminated by the database server. (This problem may only affect very low traffic sites.)
At the end of each request, Django closes the connection if it has reached its maximum age or if it is in an unrecoverable error state. If any database errors have occurred while processing the requests, Django checks whether the connection still works, and closes it if it doesnt. Thus, database errors affect at most one request; if the connection becomes unusable, the next request gets a fresh connection.
#### CAVEATS
Since each thread maintains its own connection, your database must support at least as many simultaneous connections as you have worker threads.
Sometimes a database wont be accessed by the majority of your views, for example because its the database of an external system, or thanks to caching. In such cases, you should set `CONN_MAX_AGE` to a low value or even `0`, because it doesnt make sense to maintain a connection thats unlikely to be reused. This will help keep the number of simultaneous connections to this database small.
The development server creates a new thread for each request it handles, negating the effect of persistent connections. Dont enable them during development.
When Django establishes a connection to the database, it sets up appropriate parameters, depending on the backend being used. If you enable persistent connections, this setup is no longer repeated every request. If you modify parameters such as the connections isolation level or time zone, you should either restore Djangos defaults at the end of each request, force an appropriate value at the beginning of each request, or disable persistent connections.
### Encoding
Django assumes that all databases use UTF-8 encoding. Using other encodings may result in unexpected behavior such as value too long errors from your database for data that is valid in Django. See the database specific notes below for information on how to set up your database correctly.
## PostgreSQL notes
Django supports PostgreSQL 9.0 and higher. It requires the use of Psycopg2 2.0.9 or higher.
### PostgreSQL connection settings
See `HOST` for details.
### Optimizing PostgreSQLs configuration
Django needs the following parameters for its database connections:
* `client_encoding`: `'UTF8'`,
* `default_transaction_isolation`: `'read committed'` by default, or the value set in the connection options (see below),
* `timezone`: `'UTC'` when `USE_TZ` is `True`, value of `TIME_ZONE` otherwise.
If these parameters already have the correct values, Django wont set them for every new connection, which improves performance slightly. You can configure them directly in `postgresql.conf` or more conveniently per database user with [ALTER ROLE](http://www.postgresql.org/docs/current/interactive/sql-alterrole.html).
Django will work just fine without this optimization, but each new connection will do some additional queries to set these parameters.
### Isolation level
Like PostgreSQL itself, Django defaults to the `READ COMMITTED` [isolation level](http://www.postgresql.org/docs/current/static/transaction-iso.html). If you need a higher isolation level such as `REPEATABLE READ` or `SERIALIZABLE`, set it in the `OPTIONS` part of your database configuration in`DATABASES`:
~~~
import psycopg2.extensions
DATABASES = {
# ...
'OPTIONS': {
'isolation_level': psycopg2.extensions.ISOLATION_LEVEL_SERIALIZABLE,
},
}
~~~
Note
Under higher isolation levels, your application should be prepared to handle exceptions raised on serialization failures. This option is designed for advanced uses.
### Indexes for `varchar` and `text` columns
When specifying `db_index=True` on your model fields, Django typically outputs a single `CREATE INDEX`statement. However, if the database type for the field is either `varchar` or `text` (e.g., used by `CharField`,`FileField`, and `TextField`), then Django will create an additional index that uses an appropriate [PostgreSQL operator class](http://www.postgresql.org/docs/current/static/indexes-opclass.html) for the column. The extra index is necessary to correctly perform lookups that use the `LIKE`operator in their SQL, as is done with the `contains` and `startswith` lookup types.
## MySQL notes
### Version support
Django supports MySQL 5.5 and higher.
Djangos `inspectdb` feature uses the `information_schema` database, which contains detailed data on all database schemas.
Django expects the database to support Unicode (UTF-8 encoding) and delegates to it the task of enforcing transactions and referential integrity. It is important to be aware of the fact that the two latter ones arent actually enforced by MySQL when using the MyISAM storage engine, see the next section.
### Storage engines
MySQL has several [storage engines](http://dev.mysql.com/doc/refman/5.6/en/storage-engines.html). You can change the default storage engine in the server configuration.
Until MySQL 5.5.4, the default engine was [MyISAM](http://dev.mysql.com/doc/refman/5.6/en/myisam-storage-engine.html) [[1]](http://masteringdjango.com/advanced-database-management/#id6). The main drawbacks of MyISAM are that it doesnt support transactions or enforce foreign-key constraints. On the plus side, it was the only engine that supported full-text indexing and searching until MySQL 5.6.4.
Since MySQL 5.5.5, the default storage engine is [InnoDB](http://dev.mysql.com/doc/refman/5.6/en/innodb-storage-engine.html). This engine is fully transactional and supports foreign key references. Its probably the best choice at this point. However, note that the InnoDB autoincrement counter is lost on a MySQL restart because it does not remember the `AUTO_INCREMENT` value, instead recreating it as max(id)+1. This may result in an inadvertent reuse of `AutoField` values.
If you upgrade an existing project to MySQL 5.5.5 and subsequently add some tables, ensure that your tables are using the same storage engine (i.e. MyISAM vs. InnoDB). Specifically, if tables that have a`ForeignKey` between them use different storage engines, you may see an error like the following when running `migrate`:
~~~
_mysql_exceptions.OperationalError: (
1005, "Can't create table '\\db_name\\.#sql-4a8_ab' (errno: 150)"
)
~~~
| [[1]](http://masteringdjango.com/advanced-database-management/#id4) | Unless this was changed by the packager of your MySQL package. Weve had reports that the Windows Community Server installer sets up InnoDB as the default storage engine, for example. |
### MySQL DB API Drivers
The Python Database API is described in [PEP 249](https://www.python.org/dev/peps/pep-0249). MySQL has three prominent drivers that implement this API:
* [MySQLdb](https://pypi.python.org/pypi/MySQL-python/1.2.4) is a native driver that has been developed and supported for over a decade by Andy Dustman.
* [mysqlclient](https://pypi.python.org/pypi/mysqlclient) is a fork of `MySQLdb` which notably supports Python 3 and can be used as a drop-in replacement for MySQLdb. At the time of this writing, this is the recommended choice for using MySQL with Django.
* [MySQL Connector/Python](http://dev.mysql.com/downloads/connector/python) is a pure Python driver from Oracle that does not require the MySQL client library or any Python modules outside the standard library.
All these drivers are thread-safe and provide connection pooling. `MySQLdb` is the only one not supporting Python 3 currently.
In addition to a DB API driver, Django needs an adapter to access the database drivers from its ORM. Django provides an adapter for MySQLdb/mysqlclient while MySQL Connector/Python includes [its own](http://dev.mysql.com/doc/refman/5.6/en/connector-python-info.html).
#### MYSQLDB
Django requires MySQLdb version 1.2.1p2 or later.
Note
If you see `ImportError: cannot import name ImmutableSet` when trying to use Django, your MySQLdb installation may contain an outdated `sets.py` file that conflicts with the built-in module of the same name from Python 2.4 and later. To fix this, verify that you have installed MySQLdb version 1.2.1p2 or newer, then delete the `sets.py` file in the MySQLdb directory that was left by an earlier version.
Note
There are known issues with the way MySQLdb converts date strings into datetime objects. Specifically, date strings with value 0000-00-00 are valid for MySQL but will be converted into None by MySQLdb.
This means you should be careful while using loaddata/dumpdata with rows that may have 0000-00-00 values, as they will be converted to None.
Note
At the time of writing, the latest release of MySQLdb (1.2.4) doesnt support Python 3\. In order to use MySQLdb under Python 3, youll have to install `mysqlclient`.
#### MYSQLCLIENT
Django requires [mysqlclient](https://pypi.python.org/pypi/mysqlclient) 1.3.3 or later. Note that Python 3.2 is not supported. Except for the Python 3.3+ support, mysqlclient should mostly behave the same as MySQLDB.
#### MYSQL CONNECTOR/PYTHON
MySQL Connector/Python is available from the [download page](http://dev.mysql.com/downloads/connector/python/). The Django adapter is available in versions 1.1.X and later. It may not support the most recent releases of Django.
### Time zone definitions
If you plan on using Djangos timezone support , use [mysql_tzinfo_to_sql](http://dev.mysql.com/doc/refman/5.6/en/mysql-tzinfo-to-sql.html) to load time zone tables into the MySQL database. This needs to be done just once for your MySQL server, not per database.
### Creating your database
You can [create your database](http://dev.mysql.com/doc/refman/5.6/en/create-database.html) using the command-line tools and this SQL:
~~~
CREATE DATABASE <dbname> CHARACTER SET utf8;
~~~
This ensures all tables and columns will use UTF-8 by default.
#### COLLATION SETTINGS
The collation setting for a column controls the order in which data is sorted as well as what strings compare as equal. It can be set on a database-wide level and also per-table and per-column. This is[documented thoroughly](http://dev.mysql.com/doc/refman/5.6/en/charset.html) in the MySQL documentation. In all cases, you set the collation by directly manipulating the database tables; Django doesnt provide a way to set this on the model definition.
By default, with a UTF-8 database, MySQL will use the `utf8_general_ci` collation. This results in all string equality comparisons being done in a *case-insensitive* manner. That is, `"Fred"` and `"freD"` are considered equal at the database level. If you have a unique constraint on a field, it would be illegal to try to insert both `"aa"` and `"AA"` into the same column, since they compare as equal (and, hence, non-unique) with the default collation.
In many cases, this default will not be a problem. However, if you really want case-sensitive comparisons on a particular column or table, you would change the column or table to use the `utf8_bin` collation. The main thing to be aware of in this case is that if you are using MySQLdb 1.2.2, the database backend in Django will then return bytestrings (instead of unicode strings) for any character fields it receive from the database. This is a strong variation from Djangos normal practice of *always* returning unicode strings. It is up to you, the developer, to handle the fact that you will receive bytestrings if you configure your table(s) to use `utf8_bin` collation. Django itself should mostly work smoothly with such columns (except for the`contrib.sessions` `Session` and `contrib.admin` `LogEntry` tables described below), but your code must be prepared to call `django.utils.encoding.smart_text()` at times if it really wants to work with consistent data Django will not do this for you (the database backend layer and the model population layer are separated internally so the database layer doesnt know it needs to make this conversion in this one particular case).
If youre using MySQLdb 1.2.1p2, Djangos standard `CharField` class will return unicode strings even with`utf8_bin` collation. However, [`TextField`](http://masteringdjango.com/advanced-database-management/appendix_A.html#django.db.models.TextField "django.db.models.TextField") fields will be returned as an `array.array` instance (from Pythons standard `array` module). There isnt a lot Django can do about that, since, again, the information needed to make the necessary conversions isnt available when the data is read in from the database. This problem was [fixed in MySQLdb 1.2.2](http://sourceforge.net/tracker/index.php?func=detail&aid=1495765&group_id=22307&atid=374932), so if you want to use [`TextField`](http://masteringdjango.com/advanced-database-management/appendix_A.html#django.db.models.TextField "django.db.models.TextField") with `utf8_bin` collation, upgrading to version 1.2.2 and then dealing with the bytestrings (which shouldnt be too difficult) as described above is the recommended solution.
Should you decide to use `utf8_bin` collation for some of your tables with MySQLdb 1.2.1p2 or 1.2.2, you should still use `utf8_general_ci` (the default) collation for the `django.contrib.sessions.models.Session` table (usually called `django_session`) and the `django.contrib.admin.models.LogEntry` table (usually called`django_admin_log`). Those are the two standard tables that use [`TextField`](http://masteringdjango.com/advanced-database-management/appendix_A.html#django.db.models.TextField "django.db.models.TextField") internally.
Please note that according to [MySQL Unicode Character Sets](http://dev.mysql.com/doc/refman/5.7/en/charset-unicode-sets.html), comparisons for the `utf8_general_ci` collation are faster, but slightly less correct, than comparisons for `utf8_unicode_ci`. If this is acceptable for your application, you should use `utf8_general_ci` because it is faster. If this is not acceptable (for example, if you require German dictionary order), use `utf8_unicode_ci` because it is more accurate.
Warning
Model formsets validate unique fields in a case-sensitive manner. Thus when using a case-insensitive collation, a formset with unique field values that differ only by case will pass validation, but upon calling`save()`, an `IntegrityError` will be raised.
### Connecting to the database
Refer to the settings documentation .
Connection settings are used in this order:
1. `OPTIONS`.
2. `NAME`, `USER`, `PASSWORD`, `HOST`, `PORT`
3. MySQL option files.
In other words, if you set the name of the database in `OPTIONS`, this will take precedence over `NAME`, which would override anything in a [MySQL option file](http://dev.mysql.com/doc/refman/5.6/en/option-files.html).
Heres a sample configuration which uses a MySQL option file:
~~~
# settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'OPTIONS': {
'read_default_file': '/path/to/my.cnf',
},
}
}
# my.cnf
[client]
database = NAME
user = USER
password = PASSWORD
default-character-set = utf8
~~~
Several other MySQLdb connection options may be useful, such as `ssl`, `init_command`, and `sql_mode`. Consult the [MySQLdb documentation](http://mysql-python.sourceforge.net/) for more details.
### Creating your tables
When Django generates the schema, it doesnt specify a storage engine, so tables will be created with whatever default storage engine your database server is configured for. The easiest solution is to set your database servers default storage engine to the desired engine.
If youre using a hosting service and cant change your servers default storage engine, you have a couple of options.
* After the tables are created, execute an `ALTER TABLE` statement to convert a table to a new storage engine (such as InnoDB):
~~~
ALTER TABLE <tablename> ENGINE=INNODB;
~~~
This can be tedious if you have a lot of tables.
* Another option is to use the `init_command` option for MySQLdb prior to creating your tables:
~~~
'OPTIONS': {
'init_command': 'SET storage_engine=INNODB',
}
~~~
This sets the default storage engine upon connecting to the database. After your tables have been created, you should remove this option as it adds a query that is only needed during table creation to each database connection.
### Table names
There are [known issues](http://bugs.mysql.com/bug.php?id=48875) in even the latest versions of MySQL that can cause the case of a table name to be altered when certain SQL statements are executed under certain conditions. It is recommended that you use lowercase table names, if possible, to avoid any problems that might arise from this behavior. Django uses lowercase table names when it auto-generates table names from models, so this is mainly a consideration if you are overriding the table name via the [`db_table`](http://masteringdjango.com/advanced-database-management/appendix_A.html#django.db.models.Options.db_table "django.db.models.Options.db_table") parameter.
### Savepoints
Both the Django ORM and MySQL (when using the InnoDB storage engine) support database savepoints.
If you use the MyISAM storage engine please be aware of the fact that you will receive database-generated errors if you try to use the savepoint-related methods of the transactions API. The reason for this is that detecting the storage engine of a MySQL database/table is an expensive operation so it was decided it isnt worth to dynamically convert these methods in no-ops based in the results of such detection.
### Notes on specific fields
#### CHARACTER FIELDS
Any fields that are stored with `VARCHAR` column types have their `max_length` restricted to 255 characters if you are using `unique=True` for the field. This affects `CharField`, [`SlugField`](http://masteringdjango.com/advanced-database-management/appendix_A.html#django.db.models.SlugField "django.db.models.SlugField") and `CommaSeparatedIntegerField`.
#### FRACTIONAL SECONDS SUPPORT FOR TIME AND DATETIME FIELDS
MySQL 5.6.4 and later can store fractional seconds, provided that the column definition includes a fractional indication (e.g. `DATETIME(6)`). Earlier versions do not support them at all. In addition, versions of MySQLdb older than 1.2.5 have [a bug](https://github.com/farcepest/MySQLdb1/issues/24) that also prevents the use of fractional seconds with MySQL.
Django will not upgrade existing columns to include fractional seconds if the database server supports it. If you want to enable them on an existing database, its up to you to either manually update the column on the target database, by executing a command like:
~~~
ALTER TABLE `your_table` MODIFY `your_datetime_column` DATETIME(6)
~~~
or using a `RunSQL` operation in a `data migration <data-migrations>`.
By default, new `DateTimeField` or `TimeField` columns are now created with fractional seconds support on MySQL 5.6.4 or later with either mysqlclient or MySQLdb 1.2.5 or later.
#### `TIMESTAMP` COLUMNS
If you are using a legacy database that contains `TIMESTAMP` columns, you must set `USE_TZ = False <USE_TZ>` to avoid data corruption. `inspectdb` maps these columns to `DateTimeField` and if you enable timezone support, both MySQL and Django will attempt to convert the values from UTC to local time.
### Row locking with `QuerySet.select_for_update()`
MySQL does not support the `NOWAIT` option to the `SELECT ... FOR UPDATE` statement. If `select_for_update()` is used with `nowait=True` then a `DatabaseError` will be raised.
### Automatic typecasting can cause unexpected results
When performing a query on a string type, but with an integer value, MySQL will coerce the types of all values in the table to an integer before performing the comparison. If your table contains the values `'abc'`,`'def'` and you query for `WHERE mycolumn=0`, both rows will match. Similarly, `WHERE mycolumn=1` will match the value `'abc1'`. Therefore, string type fields included in Django will always cast the value to a string before using it in a query.
If you implement custom model fields that inherit from `Field` directly, are overriding `get_prep_value()`, or use `extra()` or `raw()`, you should ensure that you perform the appropriate typecasting.
## SQLite notes
[SQLite](http://www.sqlite.org/) provides an excellent development alternative for applications that are predominantly read-only or require a smaller installation footprint. As with all database servers, though, there are some differences that are specific to SQLite that you should be aware of.
### Substring matching and case sensitivity
For all SQLite versions, there is some slightly counter-intuitive behavior when attempting to match some types of strings. These are triggered when using the `iexact` or `contains` filters in Querysets. The behavior splits into two cases:
1\. For substring matching, all matches are done case-insensitively. That is a filter such as`filter(name__contains="aa")` will match a name of `"Aabb"`.
2\. For strings containing characters outside the ASCII range, all exact string matches are performed case-sensitively, even when the case-insensitive options are passed into the query. So the `iexact` filter will behave exactly the same as the `exact` filter in these cases.
Some possible workarounds for this are [documented at sqlite.org](http://www.sqlite.org/faq.html#q18), but they arent utilized by the default SQLite backend in Django, as incorporating them would be fairly difficult to do robustly. Thus, Django exposes the default SQLite behavior and you should be aware of this when doing case-insensitive or substring filtering.
### Old SQLite and `CASE` expressions
SQLite 3.6.23.1 and older contains a bug when [handling query parameters](https://code.djangoproject.com/ticket/24148) in a `CASE` expression that contains an `ELSE` and arithmetic.
SQLite 3.6.23.1 was released in March 2010, and most current binary distributions for different platforms include a newer version of SQLite, with the notable exception of the Python 2.7 installers for Windows.
As of this writing, the latest release for Windows – Python 2.7.9 – includes SQLite 3.6.21\. You can install`pysqlite2` or replace `sqlite3.dll` (by default installed in `C:\Python27\DLLs`) with a newer version from[http://www.sqlite.org/](http://www.sqlite.org/) to remedy this issue.
### Using newer versions of the SQLite DB-API 2.0 driver
Django will use a `pysqlite2` module in preference to `sqlite3` as shipped with the Python standard library if it finds one is available.
This provides the ability to upgrade both the DB-API 2.0 interface or SQLite 3 itself to versions newer than the ones included with your particular Python binary distribution, if needed.
### Database is locked errors
SQLite is meant to be a lightweight database, and thus cant support a high level of concurrency.`OperationalError: database is locked` errors indicate that your application is experiencing more concurrency than `sqlite` can handle in default configuration. This error means that one thread or process has an exclusive lock on the database connection and another thread timed out waiting for the lock the be released.
Pythons SQLite wrapper has a default timeout value that determines how long the second thread is allowed to wait on the lock before it times out and raises the `OperationalError: database is locked` error.
If youre getting this error, you can solve it by:
* Switching to another database backend. At a certain point SQLite becomes too lite for real-world applications, and these sorts of concurrency errors indicate youve reached that point.
* Rewriting your code to reduce concurrency and ensure that database transactions are short-lived.
* Increase the default timeout value by setting the `timeout` database option option:
~~~
'OPTIONS': {
# ...
'timeout': 20,
# ...
}
~~~
This will simply make SQLite wait a bit longer before throwing database is locked errors; it wont really do anything to solve them.
### `QuerySet.select_for_update()` not supported
SQLite does not support the `SELECT ... FOR UPDATE` syntax. Calling it will have no effect.
### pyformat parameter style in raw queries not supported
For most backends, raw queries (`Manager.raw()` or `cursor.execute()`) can use the pyformat parameter style, where placeholders in the query are given as `'%(name)s'` and the parameters are passed as a dictionary rather than a list. SQLite does not support this.
### Parameters not quoted in `connection.queries`
`sqlite3` does not provide a way to retrieve the SQL after quoting and substituting the parameters. Instead, the SQL in `connection.queries` is rebuilt with a simple string interpolation. It may be incorrect. Make sure you add quotes where necessary before copying a query into an SQLite shell.
## Oracle notes
Django supports [Oracle Database Server](http://www.oracle.com/) versions 11.1 and higher. Version 4.3.1 or higher of the [cx_Oracle](http://cx-oracle.sourceforge.net/)Python driver is required, although we recommend version 5.1.3 or later as these versions support Python 3.
Note that due to a Unicode-corruption bug in `cx_Oracle` 5.0, that version of the driver should not be used with Django; `cx_Oracle` 5.0.1 resolved this issue, so if youd like to use a more recent `cx_Oracle`, use version 5.0.1.
`cx_Oracle` 5.0.1 or greater can optionally be compiled with the `WITH_UNICODE` environment variable. This is recommended but not required.
In order for the `python manage.py migrate` command to work, your Oracle database user must have privileges to run the following commands:
* CREATE TABLE
* CREATE SEQUENCE
* CREATE PROCEDURE
* CREATE TRIGGER
To run a projects test suite, the user usually needs these *additional* privileges:
* CREATE USER
* DROP USER
* CREATE TABLESPACE
* DROP TABLESPACE
* CREATE SESSION WITH ADMIN OPTION
* CREATE TABLE WITH ADMIN OPTION
* CREATE SEQUENCE WITH ADMIN OPTION
* CREATE PROCEDURE WITH ADMIN OPTION
* CREATE TRIGGER WITH ADMIN OPTION
Note that, while the RESOURCE role has the required CREATE TABLE, CREATE SEQUENCE, CREATE PROCEDURE and CREATE TRIGGER privileges, and a user granted RESOURCE WITH ADMIN OPTION can grant RESOURCE, such a user cannot grant the individual privileges (e.g. CREATE TABLE), and thus RESOURCE WITH ADMIN OPTION is not usually sufficient for running tests.
Some test suites also create views; to run these, the user also needs the CREATE VIEW WITH ADMIN OPTION privilege. In particular, this is needed for Djangos own test suite.
All of these privileges are included in the DBA role, which is appropriate for use on a private developers database.
The Oracle database backend uses the `SYS.DBMS_LOB` package, so your user will require execute permissions on it. Its normally accessible to all users by default, but in case it is not, youll need to grant permissions like so:
~~~
GRANT EXECUTE ON SYS.DBMS_LOB TO user;
~~~
### Connecting to the database
To connect using the service name of your Oracle database, your `settings.py` file should look something like this:
~~~
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'xe',
'USER': 'a_user',
'PASSWORD': 'a_password',
'HOST': '',
'PORT': '',
}
}
~~~
In this case, you should leave both `HOST` and `PORT` empty. However, if you dont use a `tnsnames.ora` file or a similar naming method and want to connect using the SID (xe in this example), then fill in both `HOST` and`PORT` like so:
~~~
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.oracle',
'NAME': 'xe',
'USER': 'a_user',
'PASSWORD': 'a_password',
'HOST': 'dbprod01ned.mycompany.com',
'PORT': '1540',
}
}
~~~
You should either supply both `HOST` and `PORT`, or leave both as empty strings. Django will use a different connect descriptor depending on that choice.
### Threaded option
If you plan to run Django in a multithreaded environment (e.g. Apache using the default MPM module on any modern operating system), then you must set the `threaded` option of your Oracle database configuration to True:
~~~
'OPTIONS': {
'threaded': True,
},
~~~
Failure to do this may result in crashes and other odd behavior.
### INSERT … RETURNING INTO
By default, the Oracle backend uses a `RETURNING INTO` clause to efficiently retrieve the value of an `AutoField`when inserting new rows. This behavior may result in a `DatabaseError` in certain unusual setups, such as when inserting into a remote table, or into a view with an `INSTEAD OF` trigger. The `RETURNING INTO` clause can be disabled by setting the `use_returning_into` option of the database configuration to False:
~~~
'OPTIONS': {
'use_returning_into': False,
},
~~~
In this case, the Oracle backend will use a separate `SELECT` query to retrieve AutoField values.
### Naming issues
Oracle imposes a name length limit of 30 characters. To accommodate this, the backend truncates database identifiers to fit, replacing the final four characters of the truncated name with a repeatable MD5 hash value. Additionally, the backend turns database identifiers to all-uppercase.
To prevent these transformations (this is usually required only when dealing with legacy databases or accessing tables which belong to other users), use a quoted name as the value for `db_table`:
~~~
class LegacyModel(models.Model):
class Meta:
db_table = '"name_left_in_lowercase"'
class ForeignModel(models.Model):
class Meta:
db_table = '"OTHER_USER"."NAME_ONLY_SEEMS_OVER_30"'
~~~
Quoted names can also be used with Djangos other supported database backends; except for Oracle, however, the quotes have no effect.
When running `migrate`, an `ORA-06552` error may be encountered if certain Oracle keywords are used as the name of a model field or the value of a `db_column` option. Django quotes all identifiers used in queries to prevent most such problems, but this error can still occur when an Oracle datatype is used as a column name. In particular, take care to avoid using the names `date`, `timestamp`, `number` or `float` as a field name.
### NULL and empty strings
Django generally prefers to use the empty string () rather than NULL, but Oracle treats both identically. To get around this, the Oracle backend ignores an explicit `null` option on fields that have the empty string as a possible value and generates DDL as if `null=True`. When fetching from the database, it is assumed that a `NULL` value in one of these fields really means the empty string, and the data is silently converted to reflect this assumption.
### `TextField` limitations
The Oracle backend stores `TextFields` as `NCLOB` columns. Oracle imposes some limitations on the usage of such LOB columns in general:
* LOB columns may not be used as primary keys.
* LOB columns may not be used in indexes.
* LOB columns may not be used in a `SELECT DISTINCT` list. This means that attempting to use the`QuerySet.distinct` method on a model that includes `TextField` columns will result in an error when run against Oracle. As a workaround, use the `QuerySet.defer` method in conjunction with `distinct()` to prevent`TextField` columns from being included in the `SELECT DISTINCT` list.
## Using a 3rd-party database backend
In addition to the officially supported databases, there are backends provided by 3rd parties that allow you to use other databases with Django:
* [SAP SQL Anywhere](https://github.com/sqlanywhere/sqlany-django)
* [IBM DB2](http://code.google.com/p/ibm-db/)
* [Microsoft SQL Server](http://django-mssql.readthedocs.org/en/latest/)
* [Firebird](https://github.com/maxirobaina/django-firebird)
* [ODBC](https://github.com/lionheart/django-pyodbc/)
* [ADSDB](http://code.google.com/p/adsdb-django/)
The Django versions and ORM features supported by these unofficial backends vary considerably. Queries regarding the specific capabilities of these unofficial backends, along with any support queries, should be directed to the support channels provided by each 3rd party project.
## Integrating Django with a legacy database
While Django is best suited for developing new applications, its quite possible to integrate it into legacy databases. Django includes a couple of utilities to automate as much of this process as possible.
Once youve got Django set up, youll follow this general process to integrate with an existing database.
### Give Django your database parameters
Youll need to tell Django what your database connection parameters are, and what the name of the database is. Do that by editing the `DATABASES` setting and assigning values to the following keys for the`'default'` connection:
* `NAME`
* `ENGINE <DATABASE-ENGINE>`
* `USER`
* `PASSWORD`
* `HOST`
* `PORT`
### Auto-generate the models
Django comes with a utility called `inspectdb` that can create models by introspecting an existing database. You can view the output by running this command:
~~~
$ python manage.py inspectdb
~~~
Save this as a file by using standard Unix output redirection:
~~~
$ python manage.py inspectdb > models.py
~~~
This feature is meant as a shortcut, not as definitive model generation. See the `documentation of inspectdb<inspectdb>` for more information.
Once youve cleaned up your models, name the file `models.py` and put it in the Python package that holds your app. Then add the app to your `INSTALLED_APPS` setting.
By default, `inspectdb` creates unmanaged models. That is, `managed = False` in the models `Meta` class tells Django not to manage each tables creation, modification, and deletion:
~~~
class Person(models.Model):
id = models.IntegerField(primary_key=True)
first_name = models.CharField(max_length=70)
class Meta:
managed = False
db_table = 'CENSUS_PERSONS'
~~~
If you do want to allow Django to manage the tables lifecycle, youll need to change the [`managed`](http://masteringdjango.com/advanced-database-management/appendix_A.html#django.db.models.Options.managed "django.db.models.Options.managed") option above to `True` (or simply remove it because `True` is its default value).
### Install the core Django tables
Next, run the `migrate` command to install any extra needed database records such as admin permissions and content types:
~~~
$ python manage.py migrate
~~~
### Cleaning Up Generated Models
As you might expect, the database introspection isnt perfect, and youll need to do some light cleanup of the resulting model code. Here are a few pointers for dealing with the generated models:
1. Each database table is converted to a model class (i.e., there is a one-to-one mapping between database tables and model classes). This means that youll need to refactor the models for any many-to-many join tables into `ManyToManyField` objects.
2. Each generated model has an attribute for every field, including `id` primary key fields. However, recall that Django automatically adds an `id` primary key field if a model doesnt have a primary key. Thus, youll want to remove any lines that look like this:
~~~
id = models.IntegerField(primary_key=True)
~~~
Not only are these lines redundant, but also they can cause problems if your application will be adding*new* records to these tables.
3. Each fields type (e.g., `CharField`, `DateField`) is determined by looking at the database column type (e.g.,`VARCHAR`, `DATE`). If `inspectdb` cannot map a columns type to a model field type, it will use `TextField` and will insert the Python comment `'This field type is a guess.'` next to the field in the generated model. Keep an eye out for that, and change the field type accordingly if needed.
If a field in your database has no good Django equivalent, you can safely leave it off. The Django model layer is not required to include every field in your table(s).
4. If a database column name is a Python reserved word (such as `pass`, `class`, or `for`), `inspectdb` will append`'_field'` to the attribute name and set the `db_column` attribute to the real field name (e.g., `pass`, `class`, or`for`).
For example, if a table has an `INT` column called `for`, the generated model will have a field like this:
~~~
for_field = models.IntegerField(db_column='for')
~~~
`inspectdb` will insert the Python comment `'Field renamed because it was a Python reserved word.'` next to the field.
5. If your database contains tables that refer to other tables (as most databases do), you might need to rearrange the order of the generated models so that models that refer to other models are ordered properly. For example, if model `Book` has a `ForeignKey` to model `Author`, model `Author` should be defined before model `Book`. If you need to create a relationship on a model that has not yet been defined, you can use a string containing the name of the model, rather than the model object itself.
6. `inspectdb` detects primary keys for PostgreSQL, MySQL, and SQLite. That is, it inserts `primary_key=True`where appropriate. For other databases, youll need to insert `primary_key=True` for at least one field in each model, because Django models are required to have a `primary_key=True` field.
7. Foreign-key detection only works with PostgreSQL and with certain types of MySQL tables. In other cases, foreign-key fields will be generated as ```IntegerField``s, assuming the foreign-key column was an ``INT```column.
### Test and tweak
Those are the basic steps from here youll want to tweak the models Django generated until they work the way youd like. Try accessing your data via the Django database API, and try editing objects via Djangos admin site, and edit the models file accordingly.