SYSTRAN Translation Stylesheets: Machine Translation driven by XSLT

The continuous expansion of the content available on the Web, combined with the internationalization of exchanges that the Web allows, increase the need for multilingual publishing. However, it is impossible to handle the voluminous amounts of short-life and specialized content (such as technical support documentation, news feeds…) in the traditional human localization workflow. Alternatively, machine translation (MT) provides on-demand translation with relatively low cost. MT usage has become common at different customization levels: from “raw” translation available for free on the Web's most popular portals, to highly customized translation for specific applications.

One facet of the “how to achieve quality translation” dilemma is how MT is integrated in the “multilingual publishing” workflow. Traditionally, this is done at a very late stage of the process, where MT engines need to cope with poorly structured content. We describe in this paper how the use of SYSTRAN Translation Stylesheets (STS) leverages document structure in an MT process: XSLT exposes the semantics underlying the document structure to the MT engine, subsequently improving the accuracy of MT rule selection, as well as facilitating the interaction with the author.

We first provide a description of the SYSTRAN Translation Stylesheet and compare its advantages over classical document filters. We will also show how this new technology naturally introduces MT in the multilingual authoring and publishing workflow.

To understand the way the Translation Stylesheet operates, one needs first to understand how “traditional” document filtering functions and the means through which the translation process can be parameterized. The next section breifly describes this information, and is followed by how Translation Stylesheets are defined. We conclude this section with short examples that illustrate the power of the Translation Stylesheet driving the translation process.

With regard to MT engines, “filters” are in charge of extracting sentences from the source document and for rebuilding a target document with translated sentences. Once extracted, the sentences are sequentially processed “out of context” — in particular, it is impossible to know if a given sentence belongs to a specific part of the original document (as represented in Figure 1, “Traditional document filter: sentences are extracted from the source document, translated with unique global parameters, and re-injected in the target document”). Therefore, no local parameterization can be performed.

Figure 1. Traditional document filter: sentences are extracted from the source document, translated with unique global parameters, and re-injected in the target document

In such a workflow, the panel of available translation parameters, i.e. the different means through which one can customize the translation, include the following:

Additionally, the user can define DNT (Do Not Translate) expressions but this requires specific markup on the input text by the user.

It is important to note that the translation parameters in traditional document filters are global — each sentence of the corpus is translated with the same parameters since there is no way to differentiate sentences once they are extracted. Primary benefits from Translation Stylesheets are to use the XSLT language in order to modify locally translation options, to define dynamically DNT expressions and to introduce new types of parameterization.

<!ELEMENT par (#PCDATA|mark|typo|tag)*>                     
<!ELEMENT mark (#PCDATA|mark|typo|tag)*>
<!ELEMENT typo (#PCDATA|mark|typo|tag)*>
<!ELEMENT tag ANY>
                        

<par>
  <typo type="bold">Quick</typo>
  <mark
     action="set" type="domain"
     value="information_technology">boot</mark>!
                                    
  <tag>This <zorglub>will not be</zorglub> 
    translated.</tag>
</par>
                            

<par>
  Démarrage <typo type="bold">rapide</typo> !
  <tag>This <foobar>will not be</foobar> translated.
  </tag>
</par>
                            

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:import href="tools.xsl"/>

<xsl:template match="text()">
  <xsl:call-template name="translate_par">
    <xsl:with-param name="source" select="."/>
  </xsl:call-template>
</xsl:template>

</xsl:stylesheet>
                  

<xsl:stylesheet version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
  xmlns:h="http://www.w3.org/1999/xhtml"
  xmlns:systran="http://www.systransoft.com/xslfunctions"
  extension-element-prefixes="systran">

<xsl:import href="tools.xsl"/>

<!-- Character properties -->
<xsl:template match="h:b|h:strong" mode="preprocess">
  <typo bold="1">
    <xsl:apply-templates mode="preprocess"/>
  </typo>
</xsl:template>

<xsl:template match="typo[@bold='1']" mode="postprocess">
  <xsl:element name="strong"
                          namespace="http://www.w3.org/1999/xhtml">
    <xsl:apply-templates mode="postprocess"/> 
  </xsl:element>
</xsl:template>

<!-- Tags with content to translate -->
<xsl:template match="h:p|h:li|h:h1|h:h2|h:h3|h:h4|h:h5|h:h6">
  <xsl:copy>
    <xsl:apply-templates select="@*"/>
    <xsl:call-template name="translate_par">
      <xsl:with-param name="source" select="node()"/>
    </xsl:call-template>
  </xsl:copy>
</xsl:template>

<!-- Translate title, resolving verb/nouns ambiguities to nouns -->
<xsl:template match="title">
  <xsl:value-of select="systran:pushValue('TITLE','1')" />
  <xsl:copy>
    <xsl:apply-templates select="@*"/>
    <xsl:call-template name="translate_par">
      <xsl:with-param name="source" select="text()"/>
    </xsl:call-template>
  </xsl:copy>
  <xsl:value-of select="systran:popValue()" />
</xsl:template>
                        
<!-- @title and @alt should be translated… -->
<xsl:template match="@title|@alt">
  <xsl:call-template name="translate_par">
    <xsl:with-param name="source" select="."/>
  </xsl:call-template>
</xsl:template>

<!-- … except for acronyms or abbreviations -->
<xsl:template match="@title[name(..)='abbr' or name(..)='acronym']">
  <xsl:copy/>
</xsl:template>
                        
<!-- Keep both source and translation in result for blockquote -->
<xsl:template match="blockquote">
    <xsl:copy-of select="." />
    <xsl:copy>
      <xsl:apply-templates select="@*"/>
      <xsl:call-template name="translate_par">
        <xsl:with-param name="source" select="node()"/>
      </xsl:call-template>
    </xsl:copy>            
 </xsl:template>
                        
</xsl:stylesheet>
                  

The millions of web pages machine translated [2] per day through search engines result pages have the following characteristic: machine translation directly deals with the publishing format – most of the time poorly structured HTML – of a content that was not originally meant to be translated. Localization is triggered “on-demand” by the user and there is no connection between the publishing workflow and this localization workflow. The translation engine must deal with the publishing format and guess the structure beyond the format. A good example of this can be seen by the use of separators (like |, •…) on webpages – they are used sometimes as item separators, sometimes as formatting symbols, sometimes as list bullets… without any algorithmic way to differentiate these uses.

More over, the translation engine is using a “tolerant” (similar to quirks mode of common browsers) HTML filter to parse the content and to restore the translated text in the original format. But the formatting of the result can be imperfect since text length is not preserved and might break “visual adjustments” performed by stylesheets or manually by webmasters.

Finally, the user, using Machine Translation (MT) as a gisting tool, is most of the time not aware of the content of the page before translation; and he would not be able to parameterize the translation process accordingly. As a result of all of this, the translation quality is generally lower than possible — the translation process is too far from the document structure to deal with its semantic.

Figure 2. Traditional MT Publishing Workflow: Authoring, Publishing, MT Localization

Figure 3. Better MT Publishing Workflow: Authoring, MT Localization, Publishing

Figure 2, “Traditional MT Publishing Workflow: Authoring, Publishing, MT Localization” represents this first usage. A direct improvement of this workflow is to introduce localization before publishing (see Figure 3, “Better MT Publishing Workflow: Authoring, MT Localization, Publishing”), on the authoring format (ideally, an XML format) with the following direct benefits:

To achieve this, translation process must deal directly with the user own XML structure and turn document syntax into linguistic information used by the engine to provide a customized translation. This is performed through the use of Translation Stylesheet as described in previous section.

The workflow can be improved further by introducing a interaction between the author and the machine translation through the STS (see Figure 4, “Ideal MT Publishing Workflow: Authoring with MT in mind, MT Localization, Publishing”). In this workflow, the author is notified during the editing of the content, similarly to an active spellcheck, that some expressions are ambiguous, that some structure is too complex and might result in a bad translation, that some word is not known by the translation engine… The author can then choose to adjust the content or to annotate the document with linguistic markup which will be processed by the STS. This added information will improve the quality of the translation for any target language, without requiring the author to be familiar with this language.

Figure 4. Ideal MT Publishing Workflow: Authoring with MT in mind, MT Localization, Publishing

The use of STS presented above is restricted to the translation of XML documents [3]. However, it would be nice to be able to handle non XML content (e.g. Microsoft Word, LaTeX, PDF, e-mails…) with all the benefits of STS, since these formats are still widely used, both for authoring and publishing. A way to achieve this is to have converters to and from an XML format. We present here how to use OASIS XLIFF [XLIFF] for this purpose. XLIFF is the XML Localization Interchange File Format designed by a group of software providers, localization service providers, and localization tools providers. This format allows to extract text content from any document format, together with normalized information about document structure and character properties. After localization, the original format can be reconstructed (merge process) based on the translated strings.

To get benefit from the STS and allows same level of translation fine-tuning that STS provide for translating XML format, the filter converting source format into XLIFF should preserve as much structure from the source document as possible. For instance, through the use of the XLIFF group tag, high level document formating such as tables or lists can be represented. Use of bpt tags can reflect character properties in the source format… The semantics of those tags is then described within an STS and allows the translation engine to deal with any type of document property. Figure 5, “STS translation of non XML document through XLIFF interchange format” describes the complete workflow.

Figure 5. STS translation of non XML document through XLIFF interchange format

We have presented an innovative application of XSLT that is used to drive a complex external process with parameters: a Machine Translation system. This mechanism binds the deep structure of the source document to the translation engine options. This renders a correlation between the document syntax and its underlying linguistic structure. Conversely, one can enrich the document structure with linguistic information in order to improve the quality of machine translation results. In an optimal workflow, the author and the translation engine interact with a feedback - structure enrichment cycle.

This tool is the first step towards an authoring process that is cognizant of the increasing demands for multilingual content.

[EXSLT]

Extended XSLT. http://exslt.org/ .

[LIBXML]

The XML C parser and toolkit of Gnome. Available under the MIT license at http://www.xmlsoft.org/ .

[XLIFF]

XML Localization Interchange File Format. White-paper available at https://www.oasis-open.org/committees/download.php/3110/XLIFF-core-whitepaper_1.1-cs.pdf .

Biography

Footnotes

1. Actually, the EXSLT [EXSLT] node-set extension function is needed; this requirement would be removed when switching to XSLT 2.0.
    
2. In early 2005, the estimated number of pages translated by SYSTRAN translation engines was about 20,000,000 pages a day.
    
3. Actually, the use of libxml2/libxslt libraries [LIBXML] allows us to handle HTML documents with STS as well.