Stuff

Essentials

  • June 2019
    S M T W T F S
    « Sep    
     1
    2345678
    9101112131415
    16171819202122
    23242526272829
    30  
  • Facebook

Posts Tagged "Jekyll"

The complexities of translation and the need for dynamic variables in the build process

Listen to this post:

You can download the MP3 file or subscribe in iTunes.

I mentioned in previous posts that I was tackling translation with static site generators, and that I would circle back around on this topic to provide more detail (see Will the docs-as-code approach scale?).

Translation is a complex undertaking. In Andrew Etter’s Modern Technical Writing, he says translation projects are time-consuming and costly. To quote:

Internationalization, the process of translating documentation to other languages, is a nightmare. If you ever think you need to do it, interface with management and perform a careful cost-benefit analysis, because the process is expensive, time-consuming, error-prone, and tedious. Once you’ve arrive at what you believe is an accurate estimate for company costs, triple it. Now you have a realistic estimate.

Etter briefly describes his translation workflow using a static site generator, Sphinx. The worklow involves using gettext scripts to convert the content into POT (Portable Object Template) files, which he sends to a translation company. The translation company converts them to PO (Portable Object) files (these file formats basically facilitate separating the text into strings that translators can manage) and after finishing the translation, sends the files back. He commits them to a repo, converts the PO files to MO (Machine Object) files, and builds them again in his Sphinx project.

There are quite a few different tools, formats, workflows, and approaches for doing translation. For example, here’s how one group handles translation with Middleman, another static site generator. Their process is quite different. They set environment variables in their configuration files that the provide information to the server about which language to build. Their process involves Codeship, Heroku, submodules in Git repositories, webhooks, custom Rack apps, and more.

My scenario is a lot simpler. For some projects, we send out files to translation agencies. One translation agency requires the content to either be HTML or Microsoft Word, while another translation agency accepts Markdown, HTML, or Word. I’m not sure what the agency does with the files (I assume they convert them to another format to ingest them in their computer-assisted translation systems), but we get the files back in the same format that we sent.

Since my content is in kramdown Markdown, translating the Markdown source format would be ideal, but translating HTML isn’t a deal-breaker either. However, here I should note that just saying Markdown is the source format hardly scratches the surface. If Markdown is your only source format (and you just have a collection of static files as your source), it would be very difficult to handle translation. You need a more robust tool to handle dynamic content, which is where a static site generator like Jekyll becomes essential.

Notice I used the word “dynamic” in the last sentence. There’s somewhat of a misnomer about static site generators. In your source content, you aren’t working just with static content, because if you were, translation would be extremely difficult. In Discover Meteor, the authors explain that static is really more dynamic than we typically credit it as being. They note,

A classic Ruby or PHP app can be said to be dynamic because it can react to various parameters on each request (for example, variables passed through the URL).

True, static HTML files can’t do that. But your static site generator can still take into account parameters during the build process. In other words, static sites are only static after they have been generated. (See Three More Ways To Make Your Static Sites Smarter)

The ability to use variables and parameters in your source is essential when setting up translation to multiple languages. It’s the ability to use these parameters, variables, and other dynamic techniques during the build process – before the files become static – that allow you to account for more sophisticated scenarios like translation even though you’re using a static site generator.

My overall strategy is to translate the [dynamic] source as much as possible, so that I can easily roll in future updates and then regenerate the site as needed. I don’t want to just work with the static HTML output. If I have an update that adds a new link in the navigation, or if I add a new file and want to reference it on multiple pages, I want to be able to re-build my site with the latest updates.

To accomplish this, in my project I have multiple directories (for example, _pages_english, _pages_german, etc.) for the documentation pages – one directory for each language. I also have separate configuration files for each language (for example, configenglish.yml, configgerman.yml, etc.). Each configuration file specifies which page directories should be included in the build.

When I build a Jekyll site, Jekyll processes the build based on information in the configuration file. By making changes to the configuration file (such as specifying the project language or directories to be included), I can control the output.

For my sidebar navigation, I want to generate the navigation from the same YAML file rather than splitting my navigation into multiple files for each language, which I would then need to keep in sync. To accomplish this, in my sidebar data file, I have different instances of the title, but each entry uses the same URL and other properties. For example, a sidebar data entry might look like this:

    - title: Sample
      titleja: ガイドスタート
      titlede: Frschun Rontzahnfüllungen
      titlesp: Ejemplo
      url: /sample.html
      ref: sample

When Jekyll builds my site, there’s logic in my sidebar scripts (enforced by the language specified in the configuration file) that check whether the navigation entry contains title, titleja, titlede, or titlesp and if so, the entry gets included in the navigation using the appropriate language title.

See what I mean by static site generators not really being static? Their output is static, but they can build in dynamic ways that leverage variables, parameters, and other programming logic.

Links on the site are relative, so I can just publish the built site into a sub-directory (such as /ja/) from the base url, and the links should all work. Making links relative is actually a huge advantage when it comes to configuring your static site generator output. I don’t think you could easily push out translated sites with absolute links, since the links wouldn’t point to the language subdirectories.

For images, instead of using standard Markdown image tags, I have an image template (an include) that automatically appends a suffix on the image file name (such as “-german” or “-spanish”) based on the language specified in the configuration file. If a lang property is specified in my include template, then the template appends the language suffix onto the file name. If no lang property is specified, no suffix gets appended.

Further, if some screenshots aren’t translated, I can use conditional logic like this to filter them out of the translated outputs:

{% if site.language == "english" %}
{% include image.html file="myimagefile.png" %}
{% endif %}

I access values from my configuration file through the site namespace. So my configuration file just has a property that says this:

language: english

BTW, I realize that unless you’re already familiar with Jekyll, you will probably just glaze over these technical details. I explain them here only to reinforce the fact that you need more than just static Markdown in your source to handle a translation project.

There are some strings that are re-used in notes (tips, cautions, warnings, etc.) and in the interface (a “Submit Feedback” button, for example). I store these strings in the configuration file and then reference them with tags such as {{site.note}}. In the configuration file, that note might look like this:

alerts:
  note: Hinweis
  tip: Spitze
  warning: Warnung
  important: Wichtig

The note then uses a note include template (referenced just like the image template, except with {% include note.html content="..."%}). The template for the include contains content that looks something like this:

<div class="mynote">{{site.alerts.note}}: {{include.content}}</div>

The {{site.alerts.note}} references the value in the configuration file – in this case, since the configuration file is for the German build, the value is Hinweis. If you had other strings that you wanted to isolate like this, you could separate them out into either your configuration file or into other data files (stored in your _data folder) and reference them as variables in your code.

(BTW, these sample translations are just filler text – I haven’t actually translated them yet.)

So far, so good. But now comes the tricky part. Let’s say I have an include to re-use the same paragraph of content, and the translator accepts only HTML. My Markdown source for the re-used component looks like this:

{% include_relative myfile.md %} 

In the HTML output, the contents of myfile.md will automatically be included wherever I’ve included this note (which I wouldn’t know until I searched for all instances of this tag in my content). Won’t this result in the translator translating the same passage multiple times, which would increase the cost?

No, it shouldn’t. The idea of translation memory (standard in most translation tools) is that the translator gets prompted when a previously translated segment appears again in the text. If the translator isn’t using a computer-assisted translation tool (CAT) that provides translation memory, you probably shouldn’t be using the translator. Without translation memory, though, this would be a problem.

So the HTML files go out to the translators, they plug them into their CAT tools, translate the files, and regenerate them back into HTML or Markdown and return them to me. Now I take the translated content and insert it into my source Markdown files (not my site output), using either Markdown or HTML (Markdown file formats can process either). I can now generate my site from the source into multiple formats.

Theoretically this should work, though I say theoretically because I haven’t pushed my approach through the whole workflow yet.

Also, the scenario that I’ve outlined at a very high level here is just the best-case scenario. In real life, there are additional complications and quirks that I have to account for.

While static site generators are flexible and allow me to implement hacks to get around quirks or oddities in non-standard systems, at some point I have to take a stance against absolute flexibility and lay down some rules, such as not allowing authors to mix relative and absolute links in the sidebar navigation, or not allowing custom include templates that don’t implement translation variables.

I spent the last few days working through my translation scenario, and I hope I’ve accounted for all the details, but I won’t know for another few months until I’ve gone through this process multiple times. I’m taking Andrew’s initial advice to triple my estimates.

I will readily admit that the more languages and formats you output to (for example, suppose I was outputting to 10 languages in both HTML and PDF), the scenario I’ve used here will get more cumbersome. At some point, it might make sense to plug into tools like Paligo that have been designed from the ground up to support robust translation workflows.

Then again, I’m betting that each system has a learning curve along with some strengths and weaknesses, and given the variety of scenarios and requirements, push button solutions might not be advantageous over more flexible and custom setups using processes like I described here.

If you’ve managed translation projects and want to share your insights, please do so in the comments below. One reason I write posts about topics or techniques that I’m still developing is to learn from others ahead of time and hopefully avoid mistakes that I might otherwise make without this input.

Back to Top | Comments Off on The complexities of translation and the need for dynamic variables in the build process

Presentation recording: Hunting for API developer documentation jobs in the San Francisco Bay area, by Andrew Davis

Presentation description

Andrew gave the the presentation “Hunting for Dev Doc Work Around the Bay” on August 15, 2016 at the STC Silicon Valley chapter in Santa Clara, California.

Here’s a description of Andrew’s presentation:

The job market for developer documentation in Silicon Valley is the hottest in the country. But developer documentation jobs can be tricky to get, and also difficult to excel in, especially if you lack an engineering background. Despite the difficulties, technical writers in these positions can bring tremendous value to engineering teams. Immersed in developer documentation, you’re constantly learning new technologies and interacting closely with engineers.

In this presentation, Andrew Davis will talk about developer documentation jobs in the Bay area, specifically:

>

  • Why is developer doc work desirable?
  • Which skills and abilities open doors into developer doc jobs?
  • Which traits and assets keep the doors open?
  • Where is demand strongest?
  • What’s the work pay?
  • How can I get there from here?

Video recording

Here’s the video recording (slides + audio):

Slides

Here are the slides. The slides have detailed speaker notes, so be sure to check those out too.

Audio

Just want the audio?

You can download the MP3 file or subscribe in iTunes.

About Andrew Davis

Andrew DavisAndrew Davis has recruited technical content developers in the SF Bay Area since 1995. He is a former software industry Technical Writer and has a reputation for both understanding and championing the role of content development.

Andrew enjoys helping those who communicate complex information get ahead by recognizing and refining their value to technology companies. He’s candid and connected and, just as importantly, he likes to help tech industry workers achieve their goals and achieve independence from intermediaries.

Andrew ran Synergistech Communications during the Internet Gold Rush years and has recently returned to solo recruiting mode, incorporating his business as Tech Comm Talent. He remains focused on recruiting great technical content development talent for discerning local technology companies. Join him on LinkedIn (http://www.linkedin.com/in/synergistech) and go to Synergistech to learn more.

Back to Top | Comments Off on Presentation recording: Hunting for API developer documentation jobs in the San Francisco Bay area, by Andrew Davis

Advanced formatting with Markdown using Jekyll and Includes

Conversation about Markdown and limitations

There’s somewhat of an interesting conversation going on about Markdown, constraints, extensibility, and limits. Eric Holscher started the conversation with Why You Shouldn’t Use “Markdown” for Documentation. Mark Baker followed it up with Why we need constrainable lightweight markup languages.

I want to add to the discussion by showing how Jekyll can be used to expand beyond Markdown’s limitations, primarily through the use of includes and page frontmatter.

Parameters with Includes

Jekyll let’s you create files in an _includes folder that you can include anywhere in your doc. For example, create a file called myfile.html. Store it in your _includes folder, and then you can embed the file anywhere by using this include syntax:

{% include myfile.html %} 

You can also pass parameters to the include, and this is where things get interesting. Suppose in myfile.html, you added this:

This is some {{include.stuff}}. 

Now stuff becomes a parameter that you can populate when you call the include. Here’s an example:

 {% include myfile.html stuff="special text" %} 

See how that works? The result will be:

This is some special text.

There’s a great Jekyll screencast on includes at Jekyll Tips.

I use includes to create templates for alerts, callouts, and images in my Jekyll Documentation theme.

Essentially you can create sophisticated HTML formatting, and only require the user to pass certain parameters in an include using the basic include code.

Parameters in Frontmatter

You can also populate your page’s frontmatter with specific properties and then pass those properties into HTML stored in a layout. This is the approach I took with the workflow maps that I wrote about here and here.

For example, the frontmatter for one of the user map pages contains the following:

---
title: Sample 1 Topic
keywords: sample
summary: "This is just a sample topic..."
sidebar: product2_sidebar
permalink: /p2_sample1/
map: true
map_name: usermap
box_number: 1
---

The layout.html file, where this content gets pushed, contains the following:

 {% if page.map == true %}

<script>
    $(document).ready ( function(){
        $('.box{{page.box_number}}').addClass('active');
    });
</script>

{% include custom/{{page.map_name}}.html %}

{% endif %}

The values for the map, map_name, and box_number get passed into the template. Hopefully you can see how these parameters stored in the page’s frontmatter populate the HTML code.

What’s cool about this approach is the ability to separate the content from the formatting. You don’t want your authors worrying about whether they have the syntax right — you just want them to supply the right values to a template.

Overall, you can create sophisticated HTML templates and then simply make each custom value a variable that you populate either through an include parameter or frontmatter parameter.

In this way, you use Jekyll to get past the limitations of Markdown syntax. For example, with the images, it would be a pain to type this syntax every time:

<figure><a class="no_icon" target="_blank" href="http://jekyllrb.com"><img class="docimage" src="http://idratherbewriting.com/documentation-theme-jekyll/images/jekyll.png" alt="Jekyll" /></a><figcaption>This is a sample caption</figcaption></figure>

With the include approach, the syntax is much simpler:

 {% include image.html file="jekyll.png" url="http://jekyllrb.com" alt="Jekyll" caption="This is a sample caption" %"}

I like this approach because it gives me a lot more flexibility. As long as I can express the code in HTML, I can extract out the customizable values into a template for the writer to populate, keeping everything simple.

For more advanced templates, I can leverage Liquid syntax to create loops and other logic.

Conclusion

Markdown syntax is limiting. Its simplicity is also its strength, since it allows larger numbers of people to easily learn and implement the syntax. However, when you need more advanced syntax, rather than trying to come up with a new Markdown variant, consider trying to create includes or other templates in your code, and then pass parameters into those templates.

Back to Top | Comments Off on Advanced formatting with Markdown using Jekyll and Includes

Other Stuff