Building the Workspace based on GitHub Actions

Last updated 2 months ago

Learn how to build the workspace using GitHub Actions

LightbulbWhat you'll learn

  • Create a workflow in GitHub
  • Creating secrets
  • Building the Blueprint workspace

Person reading a bookPrerequisites

  • Development experience
  • Access to CoreMedia GitHub, Nexus, and NPM repositories

WristwatchTime matters

Reading time: 15 to 20 minutes

Person in front of a laptopShould I read this?

This guide is for Developers.

Introduction

For a self-managed CoreMedia Content Cloud installation, you can set up a Continuous Integration (CI) infrastructure, for example, with the following tools:

  • Jenkins

  • Bamboo

  • TeamCity

  • CircleCI

  • GitHub Actions

These tools are server applications that provide build management and automated deployment to Development, Test, and Production servers.

Administrators or DevOps configure so-called ‘pipelines’ or ‘workflows’ to perform automated jobs that can be triggered manually or automatically.

This guide configures the commonly used workflow to build the complete workspace using GitHub Actions, but the same principles can be applied to other CI/CD tools. This way, you can validate whether the workspace builds successfully.

In the second part of this guide Building the Workspace with Docker Images in the CI you will learn how to build artifacts that can be uploaded to repositories.

Before you continue, however, we recommend reading GitHub’s documentation which explains the platform, the terminology, and the way it works very nicely: https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions

Prerequisites:

  • A registered account with CoreMedia for CoreMedia’s public GitHub, Nexus, and NPM repositories. If you are not registered yet, please visit https://www.coremedia.com/support and register using the form (don’t forget to provide your business email-address and your GitHub handle). You should receive an email within the next business day or earlier.

  • The GitHub repository with the CoreMedia Workspace (Version 2110 and later) was either cloned or manually created. If you have not yet created your own GitHub repository with the CoreMedia Content Cloud Blueprint workspace, visit https://releases.coremedia.com/cmcc-11/overview/distributions and either fork the workspace from the provided GitHub link or download and manually upload the workspace into your repository.

Building the CoreMedia Workspace

As the first step, you need to log into the GitHub account that hosts the cloned private repository. For this guide, we forked the repository directly, which means that the “master” branch is an overview branch, and the actual software is committed under the branch called ‘cmcc-11-2201’. This structure makes it easy for this guide to focus on the GitHub Actions and not the Blueprint code. This structure is also recommended since it can make working with multiple development streams easier during upgrade efforts.

Your vanilla setup might look like this:

workspace github
Don’t worry if you submitted your code to the master branch. This will work as well, but you need to adapt the script you will create.

Create and Configure the Workflow

  1. Change to the Actions tab.

    As you can see, GitHub provides a ton of prefabricated Actions. There is a vast marketplace, and many of them can be very helpful for other pipelines or Actions you have in mind. This is undoubtedly an advantage of GitHub, but you can also use the available actions as a knowledge base for pipelines with other CI/CD tools.

  2. In this case, you will use a Simple Workflow (1). Click on the button Configure for this Action.

    Configure the Action

    This will create a new folder in your master branch called ".github/workflows" (2) and a new YAML file "blank.yml" (3). The file contains a "Hello World" script (4) with many comments explaining the syntax.

    get started
  3. Rename the script for your case to "build-workspace.yml" (5). The title is up to you, but it’s recommended to use a short, clear, and descriptive title.

    renaming
  4. Next, you start with the first section of the YAML file. This contains the properties "name" and "on". The name is a short description of what the action is doing, and under "on", you define how and when the Action is triggered.

  5. A GitHub event triggers a GitHub workflow with GitHub Actions. There are various other options, and we recommend reading through GitHub’s documentation for GitHub events for more details and parameters: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows. In our example case, you will trigger the build immediately after any code commit.

  6. You also allow with "workflow-dispatch" that a build can be started manually via the GitHub UI.

  7. Next, continue with the "jobs"-section – basically the executed script. For your build, add one area under jobs called "build" (the name is arbitrary)

  8. Add a "name", which is a short description that is certainly helpful if you run multiple jobs, and also tells the Action on which OS the action should be executed.

  9. At this time, GitHub Actions supports Windows, macOS, and Ubuntu OS; you will use Ubuntu (with the latest version) (6) since that is recommended by and the most often used CI infrastructure OS for CoreMedia.

    ubuntu latest

Defining the Build Steps

Start with a basic configuration:

  1. Check out the code and set up Java. This is needed because every GitHub Action job runs on a freshly prepared virtual server – blank – with only the OS. Therefore, you need to ensure that the code is checked out on this virtual server and that the required JDK v11 is installed.

    name: Build full CoreMedia Workspace (CMCC 11)
    
    on:
      push:
        branches: [ master ]
      workflow_dispatch:
    
    jobs:
      build:
        name: Build
        runs-on: ubuntu-latest
    
        steps:
          - name: Checkout
            uses: actions/checkout@v2
            with:
              ref: cmcc-11-2201
              submodules: recursive
          - name: Set up JDK 11
            uses: actions/setup-java@v2
            with:
              java-version: '11'
              distribution: 'adopt'
              server-id: coremedia.external.releases
              server-username: NEXUS_USER
              server-password: NEXUS_PASSWORD
  2. Notice line 18 in the script above "ref: cmcc-11-2201"? This is the name of the branch with the actual Blueprint code you use for this guide. Adjust this value to your branch name or use "master" if you submitted the code to the main branch.

  3. You are also configuring the username and password to access Nexus. Nexus is the repository that stores the required (external as well as CoreMedia internal) artifacts for the build. Have a look at the pom.xml file on the root level of your CoreMedia Blueprint workspace. You will see the section that describes where to find this repository.

    <repositories>
      <repository>
        <id>coremedia.external.releases</id>
        <name>CoreMedia CMS</name>
        <url>https://repository.coremedia.com/nexus/content/groups/cms/</url>
        <layout>default</layout>
        <snapshots>
          <enabled>false</enabled>
        </snapshots>
      </repository>
    </repositories>
    For this step, it is essential that you have registered an account with CoreMedia Support. You will need the account to access the repository. Please visit https://www.coremedia.com/support.
  4. The next step improves the performance of the Action by establishing a cache configuration for the Maven artifacts that are built.

    - name: Cache Maven Repo
      uses: actions/cache@v2
            with:
              path: ~/.m2
              key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
              restore-keys: ${{ runner.os }}-m2
  5. You need to take care of three more tools for the build: Node.js, Sencha Command, and PNPM. We recommend to start with Node. The default Action provided by GitHub is just fine, but make sure you are using NodeJS version supported by CoreMedia!

    - name: Setup NodeJS
      uses: actions/setup-node@v2
            with:
              node-version: 16.x
  6. Next, you must install Sencha Command and configure a cache. You can download the installer for Sencha Command via ZIP or shell script based on the OS used. Also, configure an installation directory and the JVM with the flags Xmx (maximum memory allocation pool) and Xms (initial memory allocation pool).

    - name: Cache ext
      uses: actions/cache@v2
      with:
        path: |
          sencha-cmd-download
          sencha-cmd
        key: ${{ runner.OS }}-node-${{ hashFiles('apps/studio-client/**/package.json') }}
        restore-keys: |
          ${{ runner.OS }}-node-
          ${{ runner.OS }}-
    
    - name: Download and Install Sencha Command
      run: |
        mkdir -p sencha-cmd-download
        if (test ! -f sencha-cmd-download/senchacmd.zip); then curl -s -L -o sencha-cmd-download/senchacmd.zip http://cdn.sencha.com/cmd/7.2.0.84/no-jre/SenchaCmd-7.2.0.84-linux-amd64.sh.zip; fi
        if (test ! -f sencha-cmd-download/senchacmd.sh); then unzip -q -d sencha-cmd-download sencha-cmd-download/senchacmd && mv sencha-cmd-download/*.sh sencha-cmd-download/senchacmd.sh; fi
        mkdir -p sencha-cmd
        if (test ! -f sencha-cmd/sencha); then ./sencha-cmd-download/senchacmd.sh -q -dir "$PWD/sencha-cmd"; fi
        ln -s $PWD/sencha-cmd/sencha /usr/local/bin/sencha && \
        chmod -R 777 $PWD/sencha-cmd
        echo "-Xms128m" > $PWD/sencha-cmd/sencha.vmoptions
        echo "-Xmx2048m" >> $PWD/sencha-cmd/sencha.vmoptions
  7. Lastly, you need to set up and install PNPM – the package manager used. Similar to Maven artifacts, which are stored in a Nexus repository, Node artifacts are stored in an NPM Registry. You should have access to this repository with your CoreMedia account. The workflow script explicitly requests a Version 6 for NPM (see line 70); please make sure it matches the required version for CoreMedia if you run this guide with a version newer than CMCC 2201. Also, did you notice the configuration of the working directory? This is needed to build the Studio Client application. If you also want to build the Frontend Themes in another action, copy the step and point the parameter for ‘working-directory’ to "frontend".

    - name: Setup PNPM
      run: |
        NPM_AUTH_TOKEN=$(curl -s -H "Accept: application/json" -H "Content-Type:application/json" -X PUT --data '{"name": "${{ secrets.NPM_AUTH_USER }}", "password": "${{ secrets.NPM_AUTH_PASSWORD }}"}' https://npm.coremedia.io/-/user/org.couchdb.user:${{ secrets.NPM_AUTH_USER }} | jq -r .token)
        echo "::add-mask::$NPM_AUTH_TOKEN"
        echo "NPM_CONFIG_//npm.coremedia.io/:_authToken=$NPM_AUTH_TOKEN" >> $GITHUB_ENV
        echo "//npm.pkg.github.com/:_authToken=$NPM_AUTH_TOKEN" >> ~/.npmrc
        npm install -g pnpm@6
    
    - name: Install PNPM Dependencies
      working-directory: apps/studio-client
      env:
        NPM_CONFIG_@coremedia:registry: 'https://npm.coremedia.io'
        NPM_CONFIG_@jangaroo:registry: 'https://npm.coremedia.io'
      run: |
        which pnpm
        pnpm --version
        pnpm install

Building the Workspace

All prerequisite work is done now! Build the workspace.

You will do this in two steps – one build for the Studio Client and one for the Maven workspace with all other components (except the frontend themes). With all the previous configurations done, these are very quickly scripted steps.

  1. Make sure to point the working directory of the build to the Studio Client directory and provide the Username and Password for the Nexus repository with the Maven artifacts.

    - name: Build with Pnpm
      working-directory: apps/studio-client
      run: |
        pnpm -r build
        pnpm -r package
    
    - name: Build with Maven
      env:
        NEXUS_USER: ${{ secrets.NEXUS_USER }}
        NEXUS_PASSWORD: ${{ secrets.NEXUS_PASSWORD }}
      run: mvn clean install -DskipTests --builder smart -T8

    And that’s it. The full script should look like this:

    name: Build full CoreMedia Workspace (CMCC 11)
    
    on:
    push:
    branches: [ master ]
    workflow_dispatch:
    
    jobs:
    build:
    name: Build
    runs-on: ubuntu-latest
    
        steps:
          - name: Checkout
            uses: actions/checkout@v2
            with:
              ref: cmcc-11-2201
              submodules: recursive
          - name: Set up JDK 11
            uses: actions/setup-java@v2
            with:
              java-version: '11'
              distribution: 'adopt'
              server-id: coremedia.external.releases
              server-username: NEXUS_USER
              server-password: NEXUS_PASSWORD
          - name: Cache Maven Repo
            uses: actions/cache@v2
            with:
              path: ~/.m2
              key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
              restore-keys: ${{ runner.os }}-m2
    
    - name: Setup NodeJS
    uses: actions/setup-node@v2
    with:
    node-version: 16.x
    
    - name: Cache ext
    uses: actions/cache@v2
    with:
    path: |
    sencha-cmd-download
    sencha-cmd
    key: ${{ runner.OS }}-node-${{ hashFiles('apps/studio-client/**/package.json') }}
    restore-keys: |
    ${{ runner.OS }}-node-
    ${{ runner.OS }}-
    
    - name: Download and Install Sencha Command
    run: |
    mkdir -p sencha-cmd-download
    if (test ! -f sencha-cmd-download/senchacmd.zip); then curl -s -L -o sencha-cmd-download/senchacmd.zip http://cdn.sencha.com/cmd/7.2.0.84/no-jre/SenchaCmd-7.2.0.84-linux-amd64.sh.zip; fi
    if (test ! -f sencha-cmd-download/senchacmd.sh); then unzip -q -d sencha-cmd-download sencha-cmd-download/senchacmd && mv sencha-cmd-download/*.sh sencha-cmd-download/senchacmd.sh; fi
    mkdir -p sencha-cmd
    if (test ! -f sencha-cmd/sencha); then ./sencha-cmd-download/senchacmd.sh -q -dir "$PWD/sencha-cmd"; fi
    ln -s $PWD/sencha-cmd/sencha /usr/local/bin/sencha && \
    chmod -R 777 $PWD/sencha-cmd
    echo "-Xms128m" > $PWD/sencha-cmd/sencha.vmoptions
    echo "-Xmx2048m" >> $PWD/sencha-cmd/sencha.vmoptions
    
    - name: Setup PNPM
    run: |
    NPM_AUTH_TOKEN=$(curl -s -H "Accept: application/json" -H "Content-Type:application/json" -X PUT --data '{"name": "${{ secrets.NPM_AUTH_USER }}", "password": "${{ secrets.NPM_AUTH_PASSWORD }}"}' https://npm.coremedia.io/-/user/org.couchdb.user:${{ secrets.NPM_AUTH_USER }} | jq -r .token)
    echo "::add-mask::$NPM_AUTH_TOKEN"
    echo "NPM_CONFIG_//npm.coremedia.io/:_authToken=$NPM_AUTH_TOKEN" >> $GITHUB_ENV
    echo "//npm.pkg.github.com/:_authToken=$NPM_AUTH_TOKEN" >> ~/.npmrc
    npm install -g pnpm@6
    
    - name: Install PNPM Dependencies
    working-directory: apps/studio-client
    env:
    NPM_CONFIG_@coremedia:registry: 'https://npm.coremedia.io'
    NPM_CONFIG_@jangaroo:registry: 'https://npm.coremedia.io'
    run: |
    which pnpm
    pnpm --version
    pnpm install
    
    - name: Build with Pnpm
    working-directory: apps/studio-client
    run: |
    pnpm -r build
    pnpm -r package
    
    - name: Build with Maven
    env:
    NEXUS_USER: ${{ secrets.NEXUS_USER }}
    NEXUS_PASSWORD: ${{ secrets.NEXUS_PASSWORD }}
    run: mvn clean install -DskipTests --builder smart -T8
  2. You can now save the file with the green Start commit button on the right side.

    Start commit
  3. Add a description or a name for the commit.

    You will be redirected to the default view of the Code tab.

    code tab

    As you might remember from the On section, you configured that each commit will trigger the Action. Therefore, with this commit, the Action has started.

  4. Switch to the Actions tab at the top.

    Oops! It looks like your first build failed! Why is that? Well, you might have noticed that you used tokens at certain steps. You don’t want to hard-code usernames and passwords in the code file you just created. Hence, GitHub uses the concept of “Secrets”. You used them – for example, in the last step, where you added two environment variables:

    NEXUS_USER: ${{ secrets.NEXUS_USER }}
    NEXUS_PASSWORD: ${{ secrets.NEXUS_PASSWORD }}
  5. The values of these two properties use the GitHub secret notification, and you need to upload the actual value for these parameters. For this, switch to the Settings tab and navigate to the section Security > Secret > Actions.

    You need four secrets:

    • NPM_AUTH_USER: The username to access the NPM repository

    • NPM_AUTH_PASSWORD: The password to access the NPM repository

    • NEXUS_USER: The username to access the Nexus repository

    • NEXUS_PASSWORD: The password to access the Nexus repository

      Unless you have different configurations (for example, you are using your Nexus or your NPM registry, which is the recommended approach), you should be able to use your username and password for your CoreMedia account for the Nexus pair.

  6. To be able to download the packages from https://npm.coremedia.io, you need a GitHub access token which can be created via github.com (https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token). The token will require the following rights:

    • read:org

    • read:user

    • read:packages

  7. After creating the access token, you can use the npm client to log in to https://npm.coremedia.io, providing your GitHub username (when asked for a username) and the generated access token (when asked for a password):

    pnpm login --registry=https://npm.coremedia.io
  8. Add the four variables to your GitHub Secret list. Click on the button New repository secret in the upper right corner to create a new secret. Use the same parameter key you used in the script for the secret’s Name and your username as the Value. Click on the button Add secret.

    add secret
    Use the GitHub Access Token as well for the NPM_AUTH_PASSWORD value!
  9. You should have a list of these four entries:

    actions secrets
  10. Now switch back to the tab Actions, select your workflow, and click on Run workflow with this in place.

    run workflow
  11. After a few moments, your second attempt should appear as an additional line in the history on top.

    second line

    Click on this entry (1) to get to the detail view.

  12. Then, click on the job called Build to follow the console log.

    build
    build console log

    If you run this for the first time, completing the entire build will take several minutes. There are quite a few resources that need to be downloaded – so go and grab a coffee when you see that the two build steps (for PNPM and Maven) are reached.

    Notice how a few additional steps start with Post? These are cleanup steps that GitHub Actions is automatically adding for you. Very convenient!

  13. The green checkmark icon on the left indicates that the build was successful! Perfect!

    build success

    Congratulations on your first GitHub Actions workflow for building the CoreMedia Blueprint Workspace.

You might want to continue with the second part of this guide Building the Workspace with Docker Images in the CI were you will learn how to build artifacts that can be uploaded to repositories.

Copyright © 2022 CoreMedia GmbH, CoreMedia Corporation. All Rights Reserved.