All features

Quality Checks

Term Base

Translation History

Translation Memory

Style Guide

Translation Team Management

Professional Translation

Machine Translation

Features

For managers

For translators

For developers

For designers

For enterprises

Solutions

Help Center

Blog

Case Studies

Integrations

Resources

Pricing

Demo

Book a demo

Navigation Bar

Our goal is to provide you with in-depth knowledge regarding localization.

Learn how CMS localization can boost your online presence, expand your reach, and optimize for international success with our informative article.

CMSs: industry state and how to localize them best

Newsletter Sign Up

Localization Blog

This is the second post of my series "Do you speak <a class="zem_slink" title="UTF-8" href="http://en.wikipedia.org/wiki/UTF-8" target="_blank" rel="wikipedia">UTF-8</a>?". <a title="Do you speak UTF-8?" href="https://lingohub.com/blog/2012/04/do-you-speak-utf-8">Here </a>you can find the first article on this topic.

This article covers how <a class="zem_slink" title="Ruby (programming language)" href="http://www.ruby-lang.org/" target="_blank" rel="homepage">Ruby</a> 1.9 handles encoding internally and which tooling it provides for encoding issues. Prior to Ruby 1.9 a String was just a sequence of bytes. Calling the method size() returned the size of this byte array, not the character count. In Ruby 1.9 the <a class="zem_slink" title="Character encoding" href="http://en.wikipedia.org/wiki/Character_encoding" target="_blank" rel="wikipedia">Encoding</a> is stored along with that byte array.

As you can see in this example: You may ask for the encoding of a String. size() returns the actual character count. bytesize() gives you the actual number of bytes. How these two representations differ can be seen if you compare the codepoints and the bytes.

\[gist id=2818443\]

So this looks fine. But what to do if you want to change the encoding of a String eg. to write it to a file in a specific format. To change the encoding of a String you can use <a title="String#encode" href="http://www.ruby-doc.org/core-1.9.3/String.html#method-i-encode">String#encode</a>.
But you have to be aware that there are some pitfalls to that. There is not always a possibility to concatenate Strings having different encodings. This rule is true if one of the characters exceeds the <a class="zem_slink" title="ASCII" href="http://en.wikipedia.org/wiki/ASCII" target="_blank" rel="wikipedia">ASCII</a>-7 charset (having a byte value bigger than 127). A not allowed concatenation will lead to a Encoding::CompatibilityError. 

As mentioned in the prior post: text files do not have that meta information. There is no information about the encoding stored along with the file. The same restriction applies to source files. The effect of this issue can be seen if you change the encoding of a source file that includes non ASCII strings inside a text editor.

But not only text editors have to know the encoding of strings to store the correct byte sequence to disk. The Ruby interpreter has to know how to decode the read bytes. To add this meta information Ruby 1.9 allows to specify this information as a comment in the first line of the source code:

\[gist id=2815831\]

Speaking about files. How do I specify the encoding of a file that I want to read from disk? File.open() allows you to assign the encoding that will be used to read and encode the file data. In this example the read string will have the encoding specified at open().

\[gist id=2831484\]

In the second part of the above code example you can see that there is the possibility to specify the internal encoding of this file operation. The read string will be transcoded to that given format.

Doing this makes life really easier. For most applications it is a best practice to work with just one internal encoding. Preferably 'utf-8'.

Sources:
<ul>
<li><a href="http://yehudakatz.com/2010/05/05/ruby-1-9-encodings-a-primer-and-the-solution-for-rails/">http://yehudakatz.com/2010/05/05/ruby-1-9-encodings-a-primer-and-the-solution-for-rails/</a></li>
<li><a href="http://blog.grayproductions.net/articles/ruby_19s_string">http://blog.grayproductions.net/articles/ruby_19s_string</a></li>
<li><a href="http://nuclearsquid.com/writings/ruby-1-9-encodings/">http://nuclearsquid.com/writings/ruby-1-9-encodings/</a></li>
</ul>

In this article, we'll discuss how Ruby 1.9 manages encoding internally and the tools it provides to handle encoding issues. Looking to get more info? Visit Lingohub blog

Ruby and Encoding

Rails 3 added a security protection against <a class="zem_slink" title="Cross-site scripting" href="http://en.wikipedia.org/wiki/Cross-site_scripting" target="_blank" rel="wikipedia">Cross-site scripting</a> per default. This is a good thing because it makes your <a class="zem_slink" title="Web application" href="http://en.wikipedia.org/wiki/Web_application" target="_blank" rel="wikipedia">web application</a> safer, but it also can cause troubles when you are using html in your translations. What Rails does, is to distinguish between 'html_safe' and not 'html_safe' strings. When a string is not safe, it will be always escaped. Hence, your <a class="zem_slink" title="HTML element" href="http://en.wikipedia.org/wiki/HTML_element" target="_blank" rel="wikipedia">HTML tags</a> are shown in the text and not used as actual HTML tags.

What does that mean in your code? Let's digg in:

Let's say we want a formatted text like this:

<hr />

<h2>Lingohub (h2 tag)</h2>

can't wait to see it (text enclosed in p tag)

<hr />

Of course we use a resource file to store our text, so our application is ready to go global. And for demonstration purposes we put the HTML tags in the text (something I normally don't recommend). Here is our resource file:

\[gist id=2472073\]

All these translation have the text from above and the key is named after the method we use to display them.

And here is the code with the results:

\[gist id=2500433\]

The first line translates the string and puts it right in the view. Since the string isn't marked as html_safe, the tags are escaped using &lt; and &gt;.In doing so, Rails makes sure that when you display text from an untrusted source (e.g. user inputs), no harmful code can be embedded in your site. Line 2 uses the raw method to create a new string that is marked html_safe. Line 3 uses the html_safe method, which works like the raw method. Note, no raw! or html_safe! method exists to mark the existing string html_safe. There will be always a new string object created. Most interesting is Line 4 which shows a handy convention. When a translation title ends with "_html", Rails assumes that the text contains <a class="zem_slink" title="HTML" href="http://en.wikipedia.org/wiki/HTML" target="_blank" rel="wikipedia">HTML</a> and you don't want to escape it.

Like above mentioned, I am not a big fan of using HTML in your translation texts, but if you do, I would recommend to stick to the "_html" convention. It keeps the code cleaner and shows you right at the source if a translation text will be escaped or not.

Have fun translating!

Sources:
<ul>
<li><a href="http://yehudakatz.com/2010/02/01/safebuffers-and-rails-3-0/">http://yehudakatz.com/2010/02/01/safebuffers-and-rails-3-0/</a></li>
<li><a href="http://guides.rubyonrails.org/i18n.html#using-safe-html-translations">http://guides.rubyonrails.org/i18n.html#using-safe-html-translations</a></li>
</ul>

Here are three methods to incorporate HTML into your Rails translations. Click and learn more useful information in our blog

Three ways to use HTML in your Rails translations

In <a title="Internationalization (I18n) API in Ruby on Rails 3 (Part I)" href="https://lingohub.com/blog/2011/09/internationalization-i18n-api-in-ruby-on-rails-3-part-i">Internationalization (I18n) API in Ruby on Rails 3 (Part I)</a> and <a title="Internationalization (I18n) API in Ruby on Rails 3 (Part II)" href="https://lingohub.com/blog/2011/09/internationalization-i18n-api-in-ruby-on-rails-3-part-ii">Internationalization (I18n) API in Ruby on Rails 3 (Part II)</a> we already covered a lot of the Rails I18n functionality. In part III, we will now look at some more advanced features of I18n.

<h2>Using Different backend with I18n</h2>

The I18n ruby gem comes with backend support for resource files (more specifically .yml files). Additionally, it also provides simple approach to exchange the backend implementation using different gem/plugin depending on your requirements. Let’s examine an example where all translations are stored within database.

<h3>Using Database backend for Ruby I18n API</h3>

You can easily use your database for storing all the translations with the help of i18n_backend_database (<a href="https://github.com/dylanz/i18n_backend_database" target="_blank">https://github.com/dylanz/i18n_backend_database</a>) plugin. This plugin will store all your translations in the database instead of YAML files and provides a caching mechanism, so that the database doesn't get hit for every translation. When you tag items using i18n.t() within the code base, all of untranslated items will be noted as well as added into the database. You can translate untranslated textual content through an administrator panel provided by plugin.

Update 2011-02-01

Please note that this plugin is not yet ready for Rails 3 (see comments). Here are the steps to use this plugin for Rails 2.x:

\[gist id=801603\]

Add this lines in config/initializers/i18n.rb

\[gist id=801606\]

In config/routes.rb register admin panel routes

\[gist id=801610\]

Now, rails application can translate all the code from the database. All non-user generated text provided by the application needs to be wrapped in a call to I18n.t().

\[gist id=801637\]

Interpolation is handled by passing in key/value pairs as a value to an interpolation tag    ( {{ }} ).

\[gist id=801638\]

<h2>Translating Routes with translate_routes Gem</h2>

The Ruby gem translate_routes helps you to translate URLs to any number of different languages, even for an already working application. This gem works with Rails 3.x. and also with other Rails versions like 2.1.x, 2.2.x and 2.3.x. The Ruby gem translate_routes is working good with all type of routing definitions, which includes RESTful as well as named routes. You don't have to change your existing routing code, helpers, and so on; links work as expected - even in your tests.

Add translate_routes to your Gemfile:

\[gist id=801612\]

And let bundle do the rest: bundle install

1) Let's say you have this in your routes.rb file:

\[gist id=801614\]

You can see the available routes with the 'rake routes' task:

contact  /contact(.:format) {:controller=>"contact", :action=>"index"}

2) Now write your translations in a standard YAML file (e.g: in /config/i18n-routes.yml), including all the locales and their translations pairs:

\[gist id=801617\]

3) Append this line in your routes.rb file to activate the translations specifying the path of the translations file:

\[gist id=801618\]

4) Execute rake routes to see the new routes:

contact_es  /es/contacto(.:format) {:controller=>"contact", :action=>"index"}
contact_en  /contact(.:format)     {:controller=>"contact", :action=>"index"}

5) Include this filter in your ApplicationController:

\[gist id=801619\]

Now your application recognizes the different routes and sets the I18n.locale value in your controllers. But what about the routes generation? As you can see in the previous rake routes output, the contact_es_es_path and contact_en_us_path routing helpers have been generated and are available in your controllers and views. Additionally, a contact_path helper has been generated, which generates the routes according to the current request's locale.

This means that if you use named routes, you don't need to modify your application links because the routing helpers are automatically adapted to the current locale.

6) What about tests?

Of course, functional and integration testing involves some requests. The plugin includes some code to add a default locale parameter so they can remain untouched. Append it to your test_helper and it will be applied.

You can find additional information in the <a href="http://ruby-i18n.org/wiki">translate_routes' wiki</a>.

Globalize2 - Alternatives for Ruby I18n Gem

The Ruby gem Globalize2 was very popular in Rails community because it provides full-fledged internationalization solutions for Ruby on Rails application. You can use most of the features/tools independently and also you can combine them with other libraries or plugins.

The Feature/Tools for this gem are listed below. <ul> <li>Model translations – This functionality transparently translates ActiveRecord data</li> <li>Locale LoadPath – This gem load translation data from standard locations enforcing conventions that suite your needs</li> <li>Locale Fallbacks – This feature make sure your translation lookups fall back transparently through a path of alternative locales that make sense for any given locale in your application</li> <li>Translation value objects – You can access useful meta data information on the translations returned from your backend and/or translated models</li> </ul>

<h2>Conclusion</h2>

The I18n API comes with plenty of features and yet hides the complexity behind a simple to use API. With this gem, a Ruby on Rails application has a great built-in support for multiple languages. As the diversity of users is growing and more and more users speak different languages, a good localization and internationalizationsupport is necessary for every serious Web Framework. If your application has a more specific need, have a look at the various plugins and extensions available. Rails has also a great documentation, and of course there is an Internationalization section, <a href="http://rails-i18n.org/wiki">I18n Wiki</a>.

This article is the third part of the Ruby series, where we will delve into some of the more advanced features of I18n. Read it and the previous two parts in our blog

Internationalization (I18n) API in Ruby on Rails 3 (Part III)

In the <a title="Internationalization (I18n) API in Rails 3 (Part I)" href="https://lingohub.com/blog/2011/09/internationalization-i18n-api-in-ruby-on-rails-3-part-i">Internationalization (I18n) API in Rails 3 (Part I)</a> you read about <ol> <li>Definition of Internationalization & Localization</li> <li>How i18n mechanism works in Rails</li> <li>Setup/Configuration about the i18n mechanism</li> <li>Using the I18n with interpolation</li> <li>Using the I18n with pluralization</li> </ol>

You can read the full article on <a title="Internationalization (I18n) API in Rails 3 (Part I)" href="https://lingohub.com/blog/2011/09/internationalization-i18n-api-in-ruby-on-rails-3-part-i">Internationalization (I18n) API in Rails 3 (Part I)</a>, if you have not yet read it.

Now, let's start with some other features of I18n in Ruby on Rails.

Localization of Times/Dates
As I18n API provides date/time localization feature, we can easily add localized version of timestamp in Ruby on Rails application. To use this feature, first of all we need to add our localized translation for time format. Let's change our de.yml file to add a German version.
\[gist id=785930\]

Now, we can use I18n.l or Rails #l helper to localize the Time object in our desired time format.

\[gist id=785940\]

You can add many more formats for date/time to make it work using the format that you want. You can also get a ready made version of different types of local files from <a title="rails-i18n" href="https://github.com/svenfuchs/rails-i18n">rails-i18n</a> github repository. Once you place those file(s) in config/locales/ folder, they are ready to use for localization.

Using views for translation
In Rails 2.3 a new feature named "localized views (templates)" was introduced. For example, your application has an ItemsController class with an index view. Your default index template page for ItemsController is rendered by app/views/items/index.html.erb. If you want a localized version of index page, you will just have to write your text in index.de.html.erb file and then will have to save in the same folder. By setting the locale to :de, you can force Ruby on Rails application to render the German version of that template. When the locale is set to the default locale, the generic index.html.erb view will be used.

Generally, this feature is used for large amount of static content pages, where putting this large text content in YAML or in Ruby dictionaries feels cumbersome.

Translating ActiveRecord errors
You can translate Active Record validation error messages easily. Active Record uses different namespaces which can be used to translate easily models, attributes, and/or validations.

For example, an Item model with a validates_presence_of validation for the name attribute like this:

\[gist id=785954\]

In this case :blank is the key for the error message. So, Active Record will check this key in the namespaces:

\[gist id=785955\]

After following keys in the above mentioned order the result will be:

\[gist id=785957\]

If, your models are using inheritance then the messages are looked up in the inheritance chain.

For example, you might have an Book model inheriting from Item:
\[gist id=785958\]

Then Active Record will look for messages in this order:
\[gist id=785959\]

You can also add translations for Active Record error_messages_for helper like this.

\[gist id=785961\]

Organization of resource files
If you are using default i18n library settings, you may find that all files are stored in plain-text format on the disc. Placing all translations text of your Ruby on Rails application in one file per locale could be tedious to organize. You can easily store these files in a hierarchy structure which is easy to understand for you. For example, your config/locales directory could be organized like this:

\[gist id=785965\]

By following the above way to organize resource files, you can easily differentiate model and their attribute names from text inside views, and all of this from the "defaults". In Ruby on Rails, locale files in nested dictionaries structure are not being loaded automatically. So, to make it work we have to do this settings in config/application.rb

\[gist id=785967\]

Setting the Locale from the Domain Name
In the previous article you read about how to set the locale in Ruby on Rails using application.rb settings. There are also other ways to do so. You can set the locale from the domain name where your Ruby on Rails application runs. Like www.example.com will load the content in English locale, www.example.de will load the content in German locale. This way you can use top-level domain names to set the locale for Ruby on Rails application.

The benefits of using this method are: <ol> <li>The locale setting is an obvious part of the URL.</li> <li>Visitors obviously know in which language the content of the website will be displayed.</li> <li>It is very easy to implement in Ruby on Rails.</li> <li>Search engines love the content in different languages served at different domains.</li> </ol>

You can use it in your ApplicationController:

\[gist id=785972\]

We can also set the locale from the subdomain in a very similar way:

\[gist id=785973\]

You can also use switching menu option in your application like this:

\[gist id=785974\]

assuming you would set APP_CONFIG\[:deutsch_website_url\] to some value like http://www.example.de.

As previously mentioned this method has benefits, but what if you do not want to or able to provide different localizations ("language versions") on different domains. The solution to this will be to add locale code in the URL params (or request path).

Setting the Locale from the URL Params
We can also set the locale by including it in URL params like www.example.com/items?locale=de or www.example.com/de/items.

You can use it in your ApplicationController:

\[gist id=785977\]

You can get same benefits with this method as with setting the locale from the domain name, but the only little problem is it takes bit more code to implement. Here is the code to implement it by including it in our ApplicationController:

\[gist id=785979\]

This way we can easily set/override "defaults" for url_for and helper methods dependent on it. So, every url_for dependent helper method will now add the locale in the query string like, http://example.com/?locale=de.

Now, if you want URLs to look like this http://example.com/de/items (which loads the German locale), we need to set up routes with path_prefix option this way:

\[gist id=785982\]

So, when you call items_path for :de locales, an URL like http://example.com/de/items will be generated with the German locale. But, if you do not want to force the use of a locale in your routes you can set an optional path scope like:

\[gist id=785983\]

This way when you access http://example.com/items without a locale, you will not get a routing error, which is very useful for setting default locale when one is not mentioned.

Setting the Locale from the Client Supplied Information
Sometimes, you have to set the locale from the information that client had supplied (not from the URL.) i.e. user's preferred language or by choosing the locale in your application interface and saving it to their profile.

Using Accept-Language HTTP header
Client may set Accept-Language HTTP header in their browser or other client (like curl) and request the url. By supplying the information of the locale. We can implement this scenario with this code in ApplicationController.

\[gist id=785987\]

You can also use a plugin such as Iain <a href="https://github.com/iain/http_accept_language">Hecker’s http_accept_language</a> or even <a href="http://railscasts.com/episodes/151-rack-middleware">Rack middleware</a> such as Ryan Tomayko’s locale to implement this method in production mode.

Using User Profile
In your Ruby on Rails application it is also possible that users want to set the locale for their interface by setting it from database. In user's profile, user will have to choose the locale from a dropdown list and it will get saved in the database along with other profile data. In this case, you need to override the locale value something like this.

\[gist id=785990\]

That’s it for Part II. In Part III we will check
<ul>
<li>Using different backends for I18n API</li>
<li>Translating Routes</li>
</ul>

Learn about the Internationalization (I18n) API in Rails 3, including its definition, mechanism, setup, configuration, and usage with interpolation and pluralization.