Okapi Maven Plugin

people looking at a laptop screen

Photo by Safar Safarov on Unsplash

In the work we did at Lingo24 when I was there, we made use of the Okapi Framework - well a customised fork - as a key component in our content processing.

For those who don’t know the framework, it an excellent swiss-army like toolkit for localisation (l18n)/translation (t8n), which can be used either as a set of libraries or standalone tools.

Whilst the tools available are great, often I wanted to bundle part of the process within an existing CI/CD pipeline or build. As a heavy Maven user, and with winter nights setting in, the idea came to build a Maven plugin to do just that.

The result? The Okapi Maven Plugin.

The Okapi Maven Plugin

The Okapi Maven Plugin is a Maven plugin that allows execution of common actions using the Okapi Framework within a Maven Project.

It was inspired by the excellent okapi-ant project, written by Chase Tingley, one of the core Okapi Framework team. (Hey Chase 👋🏻)

Very much designed to scratch what was an immediate itch, it currently supports performing the following actions:

  • Execute a Pipeline
  • Export a Batch Configuration (BCONF)
  • Install a Batch Configuration (BCONF)

Using the Okapi Maven Plugin

The plugin is being released on Maven Central, so to use the plug-in you can simply include it in your project’s POM file in the plugins section.

<build>
  <plugins>
    <plugin>
      <groupId>io.meikle.maven.okapi</groupId>
      <artifactId>okapi-maven-plugin</artifactId>
      <version>1.0</version>
      <configuration>
          <!-- Configuration required here -->
      </configuration>
    </plugin>
  </plugins>
</build>

Okapi Version

By default, the plugin uses the latest Okapi Framework version, which at the time of writing this post is 1.43.0, with 1.44.0 coming soon.

You can validate the version being used by running the okapi:version goal.

mvn okapi:version

This can be changed by overriding the Okapi Framework’s okapi-core and okapi-application-rainbow dependencies in the plugin configuration.

<build>
    <plugins>
        <plugin>
            <groupId>io.meikle.maven.okapi</groupId>
            <artifactId>okapi-maven-plugin</artifactId>
            <version>1.0</version>
            <configuration>
                ...
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>net.sf.okapi</groupId>
                    <artifactId>okapi-core</artifactId>
                    <version>1.41.0</version>
                </dependency>
                <dependency>
                    <groupId>net.sf.okapi.applications</groupId>
                    <artifactId>okapi-application-rainbow</artifactId>
                    <version>1.41.0</version>
                    <!-- Exclude non-required UI dependencies -->
                    <exclusions>
                        <exclusion>
                            <artifactId>*</artifactId>
                            <groupId>org.eclipse.platform</groupId>
                        </exclusion>
                    </exclusions>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

Binary Configurations

One of the main use cases for the plugin is to support the building of Binary Configurations, also known as a BCONF.

A BCONF is a single file binary bundle of a number of configuration items:

  • File Filter Settings - the settings on how a file is processed
  • File Mappings - the mapping of file extensions to file filter settings
  • Pipelines - the configurations determing the steps to be performed on the content
  • Plugin JARs - optional JAR(s) with custom code, typically file filters or steps.

The ability to bundle them together in a single BCONF them useful for sharing settings amongst colleagues, or when using the Longhorn server application for executing Okapi Framework processing.

Building BCONF files can be done from the Rainbow application interface provided by the Okapi Framework. To do this you configure the settings you want in the application and can then export a configuration.

The inverse process is also supported, allowing you to import a configuration to a particular directory so it’s contents can be used within the application.

To help with these operations, the Okapi Maven Plugin allows you to build and install BCONF files as part of your build.

okapi:export

The export goal builds and export a Binary Configuration file and can be configured to package up configuration items in a number of scenarios.

By default, the goal only has one mandatory parameter, bconf. This is the name or path to where the Binary Configuration file should be exported to.

The rest of the parameters are optional, allowing you to either package a pipeline and the filter mappings and/or plugins required to successfully execute it within Longhorn, or simply bundle a set configuration assets for sharing.

<configuration>
    <!-- Mandatory Name or Path to Export the BCONF to -->
    <bconf>export.bconf</bconf>
    <!-- Optional Pipeline to be used in the BCONF -->
    <pipeline>test.pln</pipeline>
    <!-- Optional Custom Filter Mapping(s) to be embedded in the BCONF -->
    <filterMappings> 
        <filterMapping>
            <!-- File extension to be mapped -->
            <extension>.htm</extension>
            <!-- Configuration to be used -->
            <configuration>okf_html@test</configuration>
        </filterMapping>
    </filterMappings>
    <!-- Optional Plugins to be included in the BCONF using Maven FileSet format- -->
    <!-- See https://maven.apache.org/shared/file-management/fileset.html for more details -->
    <plugins>
        <!-- Optional directory - can be absolute (include Maven variables), or relative to project baseDir -->
        <!-- defaults to project baseDir -->
        <directory>input</directory>
        <!-- Optional - Includes or Excludes -->
        <includes>
            <include>**/*.jar</include>
        </includes>
    </plugins>
    <!-- Optional output directory for the Okapi pipeline -->
    <!-- Defaults to the Maven target directory -->
    <outputDirectory>/tmp</outputDirectory>
</configuration>

okapi:install

The install goal installs a Binary Configuration file (a bconf) to a folder on the local machine so it can be used.

The configuration for this goal has two mandatory items:

  • bconf - the location of the BCONF file to install
  • outputDirectory - the folder to install the BCONF to
<configuration>
  <bconf>export.bconf</bconf>
  <outputDirectory>/tmp/install-test</outputDirectory>
</configuration>

Both paths can be absolute or relative to the Maven Project’s baseDir, which can be useful when chaining with the pipeline goal.

Pipelines

At the core of the Okapi Framework is the concept of a pipeline - a list of actions in sequence that can be applied to or more input documents. Each action is a step in Okapi parlance, which when chained together can build very powerful and complex translation workflows.

Pipelines can be developed using the user interface in Okapi Rainbow or via an XML format, with the framework supplying a significant library of out-of-the-box steps and pipelines for common actions in translation workflow. Like everything in Okapi, you can build your own steps.

To help with executing pipelines, the Okapi Maven Plugin allows you to execute pre-defined pipelines as part of your build.

okapi:execute

The execute goal executes a pipeline contained in a pipeline configuration file (a pln file) against the supplied input file(s).

By default, the goal has four mandatory parameters:

  • pipeline - the path to the pipeline file
  • inputFiles - the input files to process, defined using a FileSet
  • sourceLang - the source language name or code of the content
  • targetLang - the target language name or code of the content

There are many other configuration options available to allow you to specify additional options for customised filter configurations, plugin locations, or override default locations. These can be useful if you are embedding your own custom assets into a build.


<configuration>
    <!-- Mandatory Path to pipeline to execute -->
    <pipeline>test.pln</pipeline>
    <!-- Mandatory inputFiles using Maven FileSet format-->
    <!-- See https://maven.apache.org/shared/file-management/fileset.html for more details -->
    <inputFiles>
        <!-- Optional directory - can be absolute (include Maven variables), or relative to project baseDir -->
        <!-- defaults to project baseDir -->
        <directory>input</directory>
        <!-- Optional - Includes or Excludes -->
        <includes>
            <include>**/*.txt</include>
            <include>**/*.md</include>
        </includes>
        <excludes>
            <exclude>**/*.xml</exclude>
        </excludes>
    </inputFiles>
    <!-- Mandatory Source Language Name or Code-->
    <sourceLang>English</sourceLang>
    <!-- Mandatory Target Language Name or Code-->
    <targetLang>fr-FR</targetLang>
    <!-- Optional directory containing custom plugins -->
    <!-- Defaults to the project directory -->
    <pluginsDirectory>pluginsDir</pluginsDirectory>
    <!-- Optional Explicit Plugins using Maven FileSet format- -->
    <!-- See https://maven.apache.org/shared/file-management/fileset.html for more details -->
    <plugins>
        <!-- Optional directory - can be absolute (include Maven variables), or relative to project baseDir -->
        <!-- defaults to project baseDir -->
        <directory>input</directory>
        <!-- Optional - Includes or Excludes -->
        <includes>
            <include>**/*.jar</include>
        </includes>
    </plugins>
    <!-- Optional directory containing custom filters -->
    <!-- Defaults to the project directory -->
    <filtersDirectory>filtersDir</filtersDirectory>
    <!-- Optional output directory for the Okapi pipeline -->
    <!-- Defaults to the Maven target directory -->
    <outputDirectory>/tmp</outputDirectory>
</configuration>

Next Steps

Whilst I moved on from Lingo24 before being able to use the plugin in anger, I have continued to use it for automating actions on some personal projects. I plan to keep evolving it as and when I get a chance, so would love feedback on what could be better or additional features that would be handy.

Happy translating!