What is CleanInTranslation? Reading the title of this section, you may not even know about CleanInTranslation. It can be summarized as the garbage collector for localization workflows: If a localization workflow aborted abnormally (escalated, for example), it will leave traces within contents, which mark them as being in translation.

Technically, it runs as scheduled task within the workflow server. See Section 5.6.2.3, “Configuration and Customization” in Blueprint Developer Manual for some configuration options for this scheduled task.

Effects not running CleanInTranslation: Not performing such garbage collection, may provide irrelevant in translation state reports in CoreMedia Studio and it will block affected versions of a document being in translation from being cleaned up by processes such as DestroyIntermediateVersions or tools like cleanversions (see Section 3.13.2.3.2, “Clean Versions” in Content Server Manual). This is relevant also, when CleanInTranslation has issues while performing its clean-up tasks.

These issues, which block clean-up, are, what this section is about. It tells you:

How to detect issues?

If there are no issues with your workflow and content states, the scheduled task CleanInTranslation will regularly output reports as shown in Example 4.1, “Standard Activity in Workflow Server Log”.

[INFO] com.coremedia.translate.workflow.CleanInTranslation [] -
  Merge-Version Cleanup: Started...
[INFO] com.coremedia.translate.workflow.CleanInTranslation [] -
  Merge-Version Cleanup: Done within 11.24 ms,
  cleaned up merge versions: 0.

If things start going wrong, you will find, in addition to the standard operation report, warnings such as shown in Example 4.2, “Warning in Workflow Server Log” in Workflow Server Log.

[WARN] com.coremedia.translate.workflow.CleanInTranslation [] -
 Process[coremedia:///cap/process/2288]:
 Process is in an invalid state regarding the stored derived contents and
 their master content objects.
 See debug log for more details.
 Please consider aborting the process or repairing the content.
 Tools which might help you: `cm validate-multisite`, `cm processes`,
 `cm dump`.

Having this, it is about time to take action, as the warnings will not disappear without intervention.

How to dump workflow processes?

At some point in time during issue analysis, it may be required to get the details of an affected localization workflow process. For example, if you want to analyze a reported process without increasing the log level to DEBUG or to repair a content state to the state prior to starting the localization workflow.

This section will give you some hints, how you may analyze such a process reported to have issues.

cm dump -u admin coremedia:///cap/process/2288 --blob text

process: coremedia:///cap/process/2288
  definition: Translation (coremedia:///cap/processdefinition/8)
  properties:
    autoMergeConflicts=[]
    createdContents=[
      Content[coremedia:///cap/content/124]
    ]
    derivedContents=[
      Content[coremedia:///cap/content/124],
      Content[coremedia:///cap/content/144]
    ]
    masterContentObjects=[
      Version[coremedia:///cap/version/42/1],
      Version[coremedia:///cap/version/44/1]
    ]
    premularConfigData=Blob[content type=application/json, size=137],
    Content:
```
{
  "coremedia:///cap/content/124":
    "coremedia:///cap/version/42/1",
  "coremedia:///cap/content/144":
    "coremedia:///cap/version/44/1"
}
```
    rollbackVersions=[
      Version[coremedia:///cap/version/144/1],
    ]
[...]

When invoking cm dump similar to Example 4.3, “Dump Localization Process” you will get an output similar to Example 4.4, “Dump Output”. The output is shortened here, to the most important details required for analysis and fixing CleanInTranslation issues. Other – possibly valuable information – include involved users or timestamps, telling about the age of the workflow process and its tasks.

Given the example output, it tells you:

definition: The workflow process definition, which is affected here. You may require to analyze the implementation of this process, if things go wrong in general.

Property autoMergeConflicts: This property tells about possible conflicts when merging, for example, changes in link lists. Such conflicts sometimes trigger actions by editors, which in the end may cause issues with CleanInTranslation. Thus, in this context, it is sometimes helpful to understand possible causes of invalid states.

Property createdContents: These contents were created in derived site, as they did not exist before, but are required to receive the localization result. If aborting an affected localization workflow, you may want to delete these contents, unless they are meanwhile used otherwise, like referenced in links.

Property derivedContents: Lists all derived contents that shall receive localization results.

Property masterContentObjects: Lists all master contents that either got added explicitly or implicitly (as dependent contents) to the localization workflow. If issues are related to these master contents, you may sometimes want to call cm dump for these, too, to understand their history.

Property premularConfigData: This is an artificial workflow variable, required in CoreMedia Studio to display localization workflows. Nevertheless, its contents may help you to spare some calls to cm dump. This is because it is the only place, where you directly see the relation from derived contents to their corresponding master contents. Thus, in the example we see, that the derived content with ID 124 is derived from master content having ID 42, and derived content with ID 144 is derived from master content with ID 44.

Property rollbackVersions: During the localization process, previously existing derived contents will be edited. Possibly even at different stages, with a bunch of versions created. This property stores the state of the derived content before the localization process started. It will be used to restore the state, when a rollback is chosen during the workflow execution. In context of CleanInTranslation issues, these versions may be important if you want to manually restore a state prior to starting the affected localization workflow. Nevertheless, you should take care to analyze, if editors did not meanwhile create additional versions prior to reverting to the version given here.

Note, that the actual dump may look different for custom localization workflows. But you should be able to locate comparable information inside these custom workflows.

How do mostly normal operation looks like?

[INFO]  Merge-Version Cleanup: Started...
[DEBUG] Merge-Version Cleanup: Stage 1 consumed 39.81 ms.
[DEBUG] Blacklisted derived contents: []
[DEBUG] Processed 0 derived contents.
[DEBUG] Merge-Version Cleanup: Stage 2 consumed 56.28 μs.
[DEBUG] Merge-Version Cleanup: Stage 3 consumed 17.35 μs.
[INFO]  Merge-Version Cleanup: Done within 40.45 ms,
  cleaned up merge versions: 0.

For a well-informed start analyzing possible issues, it may be good following the advice increasing the log level to DEBUG for logger com.coremedia.translate.workflow.CleanInTranslation as suggested in the log message. Example 4.5, “Standard Activity in Workflow Server Log on Debug Level” shows the messages provided in addition to the default ones for normal operation (logger name stripped for better readability). When localization workflows are active, only the number of processed derived contents will increase.

[INFO]  Merge-Version Cleanup: Started...
[...]
[DEBUG] Processed 1 derived contents.
[DEBUG] Merge-Version Cleanup: Stage 2 consumed 81.39 μs.
[DEBUG]
  Registered merge version Version[coremedia:///cap/version/42/1]
  on content Content[coremedia:///cap/content/124]
  for possible cleanup.
[DEBUG] Merge-Version Cleanup: Stage 3 consumed 3.965 ms.
[DEBUG] Merge-Versions scheduled for possible clean-up on next iteration(s): 1
[INFO]  Merge-Version Cleanup: Done within 24.60 ms,
  cleaned up merge versions: 0.

[INFO]  Merge-Version Cleanup: Started...
[...]
[DEBUG] Processed 1 derived contents.
[DEBUG] Merge-Version Cleanup: Stage 2 consumed 85.30 μs.
[DEBUG]
  Registered merge version Version[coremedia:///cap/version/42/1]
  on content Content[coremedia:///cap/content/124]
  for possible cleanup.
[INFO]
  Cleaning up merge version Version[coremedia:///cap/version/42/1]
  on content Content[coremedia:///cap/content/124]
[DEBUG] Merge-Version Cleanup: Stage 3 consumed 6.216 ms.
[INFO]  Merge-Version Cleanup: Done within 27.01 ms,
  cleaned up merge versions: 1.

Another still normal operation is shown in Example 4.6, “Content Cleanup Preparation” and Example 4.7, “Content Cleanup Execution”: It signals, that a localization workflow got aborted (manually or by escalation) and that clean-up is scheduled for next iteration of the scheduled task. The message Registered merge version tells in the given example, that version number 1 (one) of master content with ID 42 will get its in translation state cleaned. It got locked, because it was meant to be localized towards derived content with ID 124.

In the next iteration, this version gets cleaned up and any false-positive in translation state got removed.

Side Note on Cleanup Stages: Due to some asynchronous behavior between content and workflow server, clean-up should not be performed immediately. That is why clean-up with default configuration is only performed, when a given master version is recognized at least twice being in false-positive in translation state. Note, that because of this, it may also happen, that a previously detected version to clean-up is vetoed in next iteration. This is normal operation and does not need extra awareness.

How to analyze issues?

First, a note in advance: Analyzing issues is not required to get rid of corresponding warnings from CleanInTranslation. But it is strongly recommended, as ignored issues may leave contents in an unexpected state hard to recover from later. The affected process is the only reference, which will tell you about the original states, so that you should not abort/destroy it prior to having analyzed its state.

In the end the standard recommended process is:

[DEBUG] Issues found while retrieving derived contents and merge versions
  from process coremedia:///cap/process/2288
  via strategy DefaultTranslationWorkflowDerivedContentsStrategy: issues (2) =
  [
    Issue[type = ...],
    Issue[type = ...]
  ]

As soon as the logger is set to DEBUG, CleanInTranslation will report issues similar to Example 4.8, “General Issues Report”. Each of these issues should be analyzed and addressed accordingly prior to aborting/destroying the corresponding process.

Note, that in previous CMS versions, the output differed slightly. But it provided the same or similar information.

How to fix issues?

In the following sections we will now address the various issue types, what they tell you and how to address or fix those issues.

Workflows will escalate: Any workflow affected by these issues, is not operational anymore. That is, trying to continue proceeding with the given localization workflow will trigger an escalation at some point. That is, why most of these workflows need to aborted (destroyed) once you have taken care of the issues, unless there is a way to restore the original state.

The issue types mentioned are:

The list of possible issue codes represents those, assumed to be relevant for analyzing CleanInTranslation failures. If you experience other issues codes, please contact our support team, so that they may help you.

Recommended Tools

The following sections describe command-line tools, that are recommended for analyzing and possibly fixing issues:

• cm destroy: Section 3.13.3.3, “Destroy” in Content Server Manual

  Used to get rid, for example, of affected localization workflow processes.

• cm dump: Section 3.13.1.1, “Dump” in Content Server Manual

  Used to analyze affected workflow processes and contents. Note, that it is recommended providing the full ID of involved objects such as coremedia:///cap/process/2288 or coremedia:///cap/content/124. Numeric IDs will just reference contents.

• cm processes: Section 3.5.7, “Processes” in Workflow Manual

  May be required, to see, if contents are involved in any other workflow processes.

• cm validate-multisite: Section 3.13.1.11, “Validate Multi-Site” in Content Server Manual

  Always a good tool, to analyze the overall state before and after addressing issues. As alternative, most of the issues reported here are also reported as validation issues within CoreMedia Studio. Use the corresponding library filter for locating them.

DERIVED_DELETED

Issue[
  type = DERIVED_DELETED,
  master = coremedia:///cap/version/42/1,
  derived = coremedia:///cap/content/142,
  message =
    "Derived content is deleted: Content[coremedia:///cap/content/142]"
]

The reported issue given in Example 4.9, “Issue Output for DERIVED_DELETED” signals, that the derived content that is meant to receive a localization result got deleted meanwhile.

How to fix? A possible option is restoring the affected content from recycle bin. Otherwise, the localization workflow cannot continue.

If restoring is not an option, you most likely want to try to restore the original content state prior to starting the localization workflow. For details how to make use of the affected localization workflow, see Section “How to dump workflow processes?”.

Note on older CMS versions: In previous versions, the master was reported to be always unset (null). If this is the case, you may need to cm dump the derived content instead, to see its related master content.

DERIVED_DESTROYED

Issue[
  type = DERIVED_DESTROYED,
  master = <unset>,
  derived = coremedia:///cap/content/13274,
  message =
    "Derived content is destroyed: Content[coremedia:///cap/content/13274]"
]

The reported issue given in Example 4.10, “Issue Output for DERIVED_DESTROYED” signals, that the derived content that is meant to receive a localization result got destroyed meanwhile. Note, that the master cannot be provided, as the content got destroyed. Instead, you may want to analyze the workflow details to find the related master content.

How to fix? Unless you have some backup, there is no way to restore this derived content. Thus, you most likely need to revert to the state prior to starting the affected localization process. For details how to do that, see Section “How to dump workflow processes?”. Another alternative is, to review all contents related to the workflow and try to push them towards some useful state. For older localization workflows, it may even be, that editors meanwhile manually fixed the related contents, so that there is nothing to do.

DERIVED_MASTER_MISMATCH

Issue[
  type = DERIVED_MASTER_MISMATCH,
  master = coremedia:///cap/content/42,
  derived = coremedia:///cap/content/124,
  message =
    "Unable to determine master version for derived content:
    /Sites/Chef Corp./de-DE/Editorial/Document
      <Content[coremedia:///cap/content/124]>.
    Currently set master Content[coremedia:///cap/content/42] is not
    represented within master content objects as expected."
]

The reported issue given in Example 4.11, “Issue Output for DERIVED_MASTER_MISMATCH” signals, that there is no corresponding master referenced in the variable masterContentObjects of the corresponding workflow. The given value for master is the master as referenced by the derived content.

A possible reason may be, that the master got destroyed, which would raise another issue: MASTER_DESTROYED. Other possible options are, that the workflow got started with an unexpected state regarding its derivedContents and masterContentObjects variable or that the master link within the derived content got changed meanwhile.

How to fix? Assuming a relation to MASTER_DESTROYED, the approach is similar or the same: You need to understand the state of the derived content and repair it accordingly. This may mean to delete it or to adjust master and master version property.

If the workflow got started with an irregular state, or the master link got changed meanwhile, you may instead want to analyze the workflow and cm dump the process to analyze its properties. See Section “How to dump workflow processes?” for details. The result may help to repair the content or to understand possible issues of the affected localization process definition.

DERIVED_NO_MASTER

Issue[
  type = DERIVED_NO_MASTER,
  master = <unset>,
  derived = coremedia:///cap/content/124,
  message =
    "Required master missing for derived content:
     /Sites/Chef Corp./Germany/German/Example
     <Content[coremedia:///cap/content/124]>"
]

The reported issue given in Example 4.12, “Issue Output for DERIVED_NO_MASTER” signals, that the derived content meanwhile has no corresponding master property set anymore.

How to fix? Most likely, some previous version of the derived content still references the master. Thus, you may want to restore the master link. Afterwards, the localization workflow can be completed. Prior to that, you may want to understand why the master link got deleted.

If the version that references the master is not available anymore (e.g., due to version cleanup), you may instead have a look at the cm dump results to restore the master link according to those data. For details see Section “How to dump workflow processes?”.

MASTER_DESTROYED

Issue[
  type = MASTER_DESTROYED,
  master = coremedia:///cap/version/42/1,
  derived = <unset>,
  message =
    "Master content referenced by process is destroyed:
    Version[coremedia:///cap/version/42/1]."
]

The reported issue given in Example 4.13, “Issue Output for MASTER_DESTROYED” signals, that the version or content referenced in the variable masterContentObjects of the corresponding workflow got destroyed.

The issue generally comes with an additional issue DERIVED_MASTER_MISMATCH, as a missing master also means, that the corresponding master for a given derived content cannot be found.

How to fix? Having this issue means, that there is most likely some orphaned derived content, now linking to this destroyed master. As stated, this orphaned derived content should be referenced in an issue of type DERIVED_MASTER_MISMATCH. If it is not mentioned, then the localization workflow got started with an unexpected set of master and derived content objects, which, at least here, are not related to another.

Ignoring the irregular state, the easiest way to proceed is to find the corresponding issue of type DERIVED_MASTER_MISMATCH which references master with ID 42 in the given example.

Now everything, which is left to do, is to decide, how to deal with the orphaned derived content. Possible options are to remove it or at least repair the master link and master version property. For example, if it shall have no master anymore, just clear both entries. Any other options require to understand the history of this derived content and its relation to other contents, like links referencing the derived content.