Content Server Manual / 3.3 Configuring Blob Storage

Content Server Manual / Version 2204

3.3 Configuring Blob Storage

Up to CoreMedia CMS 2006, the Content Server stored all blobs in one database table. This was sufficient for many cases, but there are scenarios where other solutions are preferable, for example:

Different access characteristics for different blobs
Very high volume of blobs
Requirements on upload and delivery speed

CoreMedia CMS now offers two methods to cope with different requirements for blob storage:

Configure the table space for blob data as described in Section 3.2.1, “Specifying Tablespaces for the Content Server”
Define different "media stores" for blobs, described in this section.

Media Stores for blobs

CoreMedia CMS supports different media stores for the storage of blobs, depending on some properties of the blob, such as

size,
MIME type
target property and content type.

The Media Store is only responsible for storing and retrieving the actual bytes. The MIME content type, garbage collection, and authentication is managed by the Content Server. CoreMedia CMS supports the following media stores:

Storing blobs in files
Storing blobs as ZIP files
Zip blobs and store them in the blobstore table
Storing blobs in different database tables

By default, blobs are still stored in the CoreMedia CMS.

If you add more media stores to the Content Servers, make sure that the media stores of the Master Live Server and all Replication Live Server are identical. Otherwise, the replicator would not be able to place the blobs into the same media store from which they were read. This would lead to inconsistencies, so that the server refuses to replicate such content.

File-based blob storage

As one option for a blob store, you can store blob contents on the file system reachable by the Content Server. The Content Server takes care that the number of files/subdirectories per directory does not grow too large. By default, files are stored in the blobstore/file directory below the Content Server installation.

For consistent backup, suspend the deletion of unused blobs by the server's blob collector as described in Section 3.8.1, “Backup Strategy” before starting the backup process.

Caution

This feature is not intended to manage existing file structures, and does not support file access that bypasses the Content Server. Also, a file structure may be accessed by only one Content Server at a time.

Storing blobs as zipped files

You can compress blobs which are not compressed by themselves, such as text blobs, and store them in the file system or in the database. By default, the zipped files are stored in the blobstore/zipfile directory below the Content Server installation directory or in the blobstore table of the database. Storing zipped files in the database requires a temporary directory which is var/temp by default.

Storing blobs in different database tables

You can configure multiple database tables, each with individual database-specific characteristics, to store the blobs in. The Content Server will not create the tables automatically, so you have to create them manually.

What happens to existing blobs

If you add a blob store for an existing Content Server chances are high that blobs are already stored in the database. These blobs will not automatically be moved to the newly configured blob store. On the Master Live Server, the blobs will be moved to the new store the next time when the content item containing the blob is published. On the Content Management Server you have to download the blob from the content item and upload it again. It's not enough to create a new content item version.

Configuring blob store selectors

Note

Not Mandatory: You only need to do this configuration, if you want to store your blobs in a location different from the default location in the CMS.

Blob storage is configured in the Spring application context of the Content Server. The predefined Spring bean blobstore of type MediaStoreConfiguration provides two customizable properties:

mediaStoreSelectors: the list of media store selectors
mediaStores: a map of media stores by their names

If you want to store blobs in different locations, depending on some properties of the blob, you can add media store selectors to the Content Server. A media store selector defines under which conditions a specific media store should be used. Example 3.4, “Example configuration” shows the configuration of two media store selectors. One for blobs larger than 1000000 bytes, which are stored in the file system, and one for images smaller than 8192 bytes, which are stored in a different database table.

<customize:prepend id="blobstoreCustomizer"
  bean="blobstore"
  property="mediaStoreSelectors">
<description> Select a media store </description>
<list>
 <!-- store big blobs in the file system. -->
      <bean class=
       "hox.corem.server.media.ConditionalMediaStoreSelector">
        <property name="storeName" value="file"/>
        <property name="condition">
          <bean class="hox.corem.server.media.MatchCondition">
            <property name="minimumLength" value="1000000"/>
          </bean>
        </property>
      </bean>

      <!-- store images of a certain minimum size in a different
      database table  -->
      <bean
      class="hox.corem.server.media.ConditionalMediaStoreSelector">
        <property name="storeName" value="dbblob"/>
        <property name="condition">
          <bean class="hox.corem.server.media.MatchCondition">
            <property name="primaryType" value="image"/>
            <property name="maximumLength" value="8192"/>
          </bean>
        </property>
      </bean>
 </list>
</customize:prepend>

Example 3.4. Example configuration

The following blob store names are predefined:

"file" for storing blob contents in the file system
"dbblob" for storing blob contents in the database table blobstore. Consult the Javadoc of hox.corem.server.media.jdbc.BlobStore for further details.
"zipfile" for storing blob contents in the file system in zipped format
"zipdbblob" for storing blob contents in the database table blobstore, in zipped format.
the empty string, "", for storing blob contents in the CMS tables as in previous releases.

Media store selectors are stored in the property mediaStoreSelectors of the blobstore bean. You can use the customizer tag <customize:prepend> to prepend your media store selectors to the list. The Content Server iterates over this list and the first matching condition defines the blob store to use for the specific blob.

You can check the effective order of custom media store selectors by looking at the INFO log output when the Content Server starts. Just search for a message containing "Configured media store selectors".

Custom media stores can be appended to the map in the mediaStores property of the blobstore bean.

The following example adds a custom media store named videostore and a media store selector to select that store for certain MIME types:

<customize:append id="addCustomBlobstore"
  bean="blobstore" property="mediaStores">
  <map>
    <entry key="videostore" value-ref="customVideoStore"/>
  </map>
</customize:append>

<customize:prepend id="addVideoStoreSelector"
  bean="blobstore"
  property="mediaStoreSelectors">
<list>
  <bean class="hox.corem.server.media.ConditionalMediaStoreSelector">
    <property name="storeName" value="videostore"/>
    <property name="condition">
      <bean class="hox.corem.server.media.OrCondition">
        <property name="conditions">
          <list>
            <bean class="hox.corem.server.media.MatchCondition">
              <property name="primaryType" value="image"/>
            </bean>
            <bean class="hox.corem.server.media.MatchCondition">
              <property name="primaryType" value="video"/>
            </bean>
            <bean class="hox.corem.server.media.MatchCondition">
              <property name="primaryType" value="audio"/>
            </bean>
            <bean class="hox.corem.server.media.MatchCondition">
              <property name="primaryType" value="application"/>
              <property name="subType" value="vnd.rn-realmedia"/>
            </bean>
          </list>
        </property>
      </bean>
    </property>
  </bean>
 </list>
</customize:prepend>

Example 3.5. Custom store configuration

For further details, see the Javadoc of package hox.corem.server.media.

If you need to, you can use the full power of Spring for configuration. However, as an administrator, you only need the following beans and tags to construct your condition when using Spring XML bean definitions.

The ConditionalMediaStoreSelector is the bean which chooses the blob store to use. It has a property condition which takes the condition.

<bean>

Child elements: <property>

Parent elements: <list>, <customize:prepend>, <customize:append>

The <bean> element defines the bean which should be configured. In the context of the blob storage media store class or condition bean, which should be configured.

Property	Description
class	Defines the Bean class, which should be modified. In the context of the blob storage, these are media store beans or condition bean, which should be configured.

Property

Description

class

Defines the Bean class, which should be modified. In the context of the blob storage, these are media store beans or condition bean, which should be configured.

Table 3.1. Attributes of the bean element

The following condition beans can be used.

Condition Name Description

hox.corem.server.media.AndCondition A condition which takes a list of conditions in the conditions property. You configure a list with the <list> element. It returns "true" when all conditions are fulfilled.

hox.corem.server.media.OrCondition A condition which takes a list of conditions in the conditions property. You configure a list with the <list> element. It returns "true" when at least one of the conditions is fulfilled.

Condition Name	Description
`hox.corem.server.media.AndCondition`	A condition which takes a list of conditions in the `conditions` property. You configure a list with the `<list>` element. It returns "true" when all conditions are fulfilled.
`hox.corem.server.media.OrCondition`	A condition which takes a list of conditions in the `conditions` property. You configure a list with the `<list>` element. It returns "true" when at least one of the conditions is fulfilled.
`hox.corem.server.media.MatchCondition`	A condition which takes some values defined with `<property>` elements and which compares these values with the values of the blob to store. The MatchCondition can take the following values: `primaryType` - Required primary MIME type of the content. `subType` - Required sub MIME type of the content. `typeParam` - Required parameter for the MIME type of the content. Can be either of the form "name=value" to test for a specific value for a type parameter, or of the form "name" (without '=') to test for presence of a "name" parameter. `minimumLength` - Required minimum content length in bytes. `maximumLength` - Required maximum content length in bytes. `documentTypeName` - Required name of the content type of the target content item. Subtypes do not match. `declaringDocumentTypeName` - Required name of the exact content type declaring the property that will receive the blob. Subtypes match only if the property was not redefined in a more specific type. `propertyName` - Required property name of the target property. `isVersion` - Required property of the version attribute. That is, will the blob be stored as a version ("true") or as content. It returns "true" when all values match.

hox.corem.server.media.MatchCondition

A condition which takes some values defined with <property> elements and which compares these values with the values of the blob to store. The MatchCondition can take the following values:

primaryType - Required primary MIME type of the content.
subType - Required sub MIME type of the content.
typeParam - Required parameter for the MIME type of the content. Can be either of the form "name=value" to test for a specific value for a type parameter, or of the form "name" (without '=') to test for presence of a "name" parameter.
minimumLength - Required minimum content length in bytes.

maximumLength - Required maximum content length in bytes.
documentTypeName - Required name of the content type of the target content item. Subtypes do not match.
declaringDocumentTypeName - Required name of the exact content type declaring the property that will receive the blob. Subtypes match only if the property was not redefined in a more specific type.
propertyName - Required property name of the target property.
isVersion - Required property of the version attribute. That is, will the blob be stored as a version ("true") or as content.

It returns "true" when all values match.

Table 3.2. Condition classes which could be used in the <bean> element.

<property>

Child elements: <bean>, <list>

Parent elements: <bean>

The <property> element defines the property of a bean, which should be configured.

Attribute	Description
name	The name of the bean property, which should be configured.
value	The value, which should be added to the bean property.

Table 3.3. Attributes of the property element

<list>

Child elements: <bean>

Parent elements: <property>

The <list> element groups conditions, which should be used in an AndCondition or OrCondition. It has no attributes.

Configuring store locations for file storage

You can change the default locations for the storage of blobs in the file system by using Spring application properties.

For the location of blobs stored in the file system use the cap.server.blobstore.file.rootdir property.
For the location of blobs zipped and stored in the file system use the cap.server.blobstore.zipfile.rootdir property.
For the location of temporary files for blobs zipped and stored in the database use the cap.server.blobstore.zipdbblob.tmpdir property.

Search Results

Table Of Contents

Filter