Starting with CoreMedia Content Cloud major version 10 (CMCC 10), this repository has been restructured to better reflect that the overall software system consists of several applications.

Overview

Since CoreMedia applications have been developed monolithically for years, there are lots of dependencies and shared code between the applications. Also, the build process of different applications was not independent, because they shared build configuration (through parent POMs).

The new CoreMedia Blueprint workspace structure is modular in the sense that it consists of many (sub-)workspaces that can be built independently, only interacting through Maven artifacts. Shared code still exists, and shared workspaces must be built before application workspaces, but workspaces of different applications are independent.

Besides shared workspaces (shared/*) and application-specific workspaces (apps/*), there are global Workspaces (global/*) that depend on several to all applications.

Workspace Concepts and Terminology

Workspaces

To reduce build-time dependencies and allow modular builds, the concept of workspaces has been introduced. A workspace is a Maven multi-module project that can be built independently, only relying on artifacts from the Maven repository, but not on anything else being present in the same Git repository. This means one Git repository hosts several workspaces. Since a workspace is a group of Maven modules and each module only belongs to one workspace, dependencies between modules of different workspaces lead to dependencies between their workspaces. In other words, workspaces are a coarsening of Maven modules and their dependencies, just like modules (and their dependencies) are a coarsening of classes (and their dependencies).

Applications (Apps) and Shared Code

CoreMedia Content Cloud is a software system that consists of several applications. Here, an application is a piece of software running in the same execution environment (usually a JVM), serving a certain (business) objective, and communicating with other applications via remote calls. Examples of CoreMedia applications are CAE, Studio Server, Studio Client (execution environment: browser!), Content Server, and all Commerce Adapters.

An application consists of one or more application-specific workspaces and reuses shared code from arbitrary many other workspaces, but not from other application workspaces. This means that all code and resources used by more than one application must not be located in an application-specific workspace, but in a shared code workspace.

Putting all shared code into one workspace would have been too coarse-grained. One has to consider that shared code changes are much more expensive, since they potentially affect any application, thus after changes, all applications have to be rebuilt, retested, redeployed, and re-released.

CoreMedia CMS has a four-tier architecture: Between frontend and persistent data storage, unlike most architectures that use one "backend" tier, CoreMedia CMS features two tiers. The backend tier consists of Content Server, Workflow Server and Search (a specifically configured Solr). The middle tier acts as a frontend façade to the backend for delivery (CAE, Headless Server), editorial interface (Studio Server, User Changes, Studio Package Proxy, Site Manager), search (CAE Feeder, Content Feeder), and other tasks (Elastic Worker).

Figure 4.1. CoreMedia CMS's Four-Tier Architecture

Analyzing code reuse between applications in our code base validated the assumption that this four-tier architecture has a major influence on code sharing. Middle-tier servers share code not reused by backend servers, and vice versa. Computing the set of shared modules, it turned out that there was only one module shared by the backend servers (cap-serverbase), so CoreMedia decided not to create a workspace with just one module and ended up with two shared code workspaces:

Global Modules

Despite the clear separation of application development, there is the need to unite all applications to a complete CoreMedia CMS software system. There are two use cases for doing so:

The system deployment workspace is called global/deployment.

All Workspaces

The following diagram shows all workspaces, grouped into shared, apps, and global.

Figure 4.2. CoreMedia CMS's Shared, Application-Specific, and Global Workspaces

Dependency Management

Putting applications into focus leads to the idea that dependency management can also be done modularly, namely for each application, because each runs in its own execution environment. Since application-specific code only runs in one execution environment, there are application-specific external dependencies that are managed centrally for each application. This means that not every (sub-)workspace needs its own external dependency management.

However, shared code needs to run in all applications that use it, so by reusing shared code, an application also inherits the shared code's dependency management.

Maven implements reused dependency management through "bill of material" (BOM) POMs. This means that there are third-party dependency management BOM POMs for each shared workspace and for each application.

Figure 4.3. Backend Tier Workspace Dependencies

Figure 4.4. Middle Tier Workspace Dependencies

Enforcer

For global management of banned dependencies a customized BannedDependencies Enforcer rule is used that reads the banned dependencies from a configuration file on the classpath. The configuration file for the CMS comes with the dependency on com.coremedia.cms:common-banned-dependencies. It is an XML file which contains the bannedDependencies configuration element that you would normally include in the configuration of the enforcer plugin. It is also possible to add additional includes and excludes directly in the custom rule element.

A new Enforcer rule called ModularOneRepoEnforcerRule has been added which mainly enforces that one workspace always manages its dependencies on other workspaces. You should always manage this kind of dependencies by importing the BOM of the other workspace instead of using a versioned dependency directly. On the other hand, for dependencies inside one workspace you should use project.version. If you have to violate these rules (for a hotfix, for instance), you can ignore certain dependencies, by adding an ignoredDependencies element to the rule, which works the same way as in the maven-dependency-plugin. The filter syntax is: [groupId]:[artifactId]:[type]:[version] where each pattern segment is optional and supports full and partial * wildcards. An empty pattern segment is treated as an implicit wildcard.

Following this pattern enables you to build the workspaces independently and to even use different versions for the separate workspaces.

Remark on Group and Artifact IDs

With the introduction of separate workspaces some aggregator and parent modules have to be copied to more than one workspace (blueprint-parent, for instance). To make the Maven coordinates unique the artifact IDs of these modules were prefixed with the name of the workspace (for example, cae.blueprint-parent), while the directory of the modules stayed as they were (for example, blueprint-parent/). The groupId could have been used for this, which would have been the more natural solution, but in order to get consistent group IDs for a workspace this would have meant new group IDs for every single artifact, which was refrained from changing for now.

There are some exceptions where the modules are copied to many workspaces, but got a real distinct artifact ID and directory (for example, cae-core-bom). These modules distinguish themselves as they are also relevant outside of a workspace, in contrast to the parents and aggregators, where the focus is put more on the similarity to the old structure and the other workspaces.

Development Use Cases

The new repository structure encourages working on a single workspace at a time, or at least on few workspaces.

Currently, you have to build workspace common and in most cases, that is when working on a middle tier app, workspace middle. Later, there should be a CI that produces SNAPSHOT artifacts for all modules from branch master, so that you can let Maven fetch artifacts from there and only do local builds of workspaces you actually work on.

Working with Application-Specific Code Only

When your task only involves one application, build common, (if it is a middle tier app) middle, and the application's workspace on the command line:

  for ws in shared/common shared/middle apps/<some-app>; do mvn clean source:jar install -f $ws -DskipTests <more-options>; done

Then, open only the application's workspace in IDEA. The goal source:jar allows browsing sources of shared code, even though they are not part of the IDEA project.

All Java applications (except Site Manager) are Spring Boot applications and can be started locally like so:

    mvn spring-boot:run -pl :<someapp>[-<variant>]-app

Adding the option -Dinstallation.host=<FQDN> connects your local application to a CI reference system. For details, see the README file in the Spring Boot folder. Alternatively, you copy the IDEA run configuration provided in the ideaRunConfiguration subfolder of the Spring Boot folder to .idea/runConfigurations, as also described in that README.

Working with Shared Code Only

This use case is quite similar to the first one.

When working with shared/middle, you have to build shared/common first.

When working with shared/common, nothing needs to be built before.

Keep in mind that changes in shared code have impact on many, sometimes even all CoreMedia applications. Treat shared code like public API!

Working with Application-Specific and Shared Code

There is still a lot of shared code, so it might happen more often than not that part of the code you must touch to implement an application feature is located in a shared workspace. The advantage of the new multi-workspace structure is that you can immediately tell that code is shared by the fact that it is located under a path starting with shared/.

The idea of modularization is to not fall into monolithic development mode (see below) just because you change shared code. In an ideal world, all shared code's contracts would be checked by unit tests. So if you change shared code in a non-breaking fashion and no tests fail, you can use new API in the application you actively work on and need not worry about other applications also using the changed code.

Even if you do not have sufficient unit tests coverage of shared code, you might have integration tests that should detect shared code changes that break other applications. Thus, if you push your shared code changes and your application-specific changes to a feature branch, your local CI should take care of validating that no other applications are (negatively) affected by your changes. Treat shared code as having an API, and you should be fine.

Working in IDEA, the most convenient way is to add the needed shared code workspace(s) to the application's IDEA project.

After that, you have to run "Reimport Maven Projects" to update the dependencies on shared code from references into your local Maven repository to references to the corresponding IDEA modules. This enables a fast development turn-around after changes in shared code, including source-level debugging and hot deploy.

Working with (Almost) All Code

If your task requires global changes, for example, a shared third-party library is updated to a new (major) version, you can still use the multi-workspace repository like the old monolithic workspace.

You can simply build the whole workspace through the root POM and then open it in IDEA.

Now, having one big IDEA project, you can do global refactorings or search and replace.

It is not recommended to work like this for normal feature implementation, because importing the large overall project into IDEA takes quite some time, and after switching to a different branch or merging in master, this process has to be repeated over and over again.

Even if you have to perform application-spanning changes, try to find a subset to work on:

The CoreMedia Blueprint workspace contains the modules and test-data top level aggregator modules.

modules

Almost every workspace, be it an application, shared or global workspace, has a modules top-level aggregator module which is the most important space for project developers. All code, resources, templates and the like is maintained here. You can start all components locally in the modules area.

The modules hierarchy consists of modules that build libraries and modules that assemble these libraries to applications. Library modules are being built with the standard Maven jar packaging type or with the swc packaging type, which is a custom packaging type to package CoreMedia Studio client libraries and plugins.

Most applications created by the modules below the modules folder are Spring Boot applications using the standard Maven jar packaging type. The CoreMedia Studio client is an Ext JS browser application with the special packaging type jangaroo-app. All other applications are command line tools built with the custom coremedia-application packaging type. coremedia-application modules are built with the coremedia-application-maven-plugin, a custom plugin tailored to the CoreMedia .jpif based application runtime.

The modules folder is structured in sub-hierarchies by grouping modules due to their functionality. There is a dedicated group cmd-tools for command line tools and functional groups like ecommerce. Since the introduction of application-oriented workspaces, the groups for these applications (cae, studio, ...) are mostly redundant, but kept for structural similarity to previous releases of CoreMedia Content Cloud. The same holds true for the group named shared, whose modules now are in most cases part of one of the two shared workspaces. The remaining two groups extension-config and extensions are required for the extensions functionality of CoreMedia Blueprint workspace.

By default, CoreMedia Blueprint workspace ships preconfigured with many extensions such as Adaptive Personalization or Elastic Social. Typically, extensions do not extend one, but many applications. CoreMedia Project Extensions decouple the application from the dependencies it is extended by and lets you automatically manage these dependencies. Not all extensions will be used in a project right from the start. In this case, the CoreMedia Extension Tool allows you to easily deactivate features that you do not need. See Section 4.1.5, “Project Extensions” for details.

test-data

The content/test-data folder contains test content to run CoreMedia Blueprint with. It can be imported into the content repository by using the CoreMedia serverimport tool. Extensions may contain additional test-data folders.