Earlier this year, my team at work and I were looking at the errors occurring in one of our signup APIs. We saw that nearly 5% of our requests were getting failed, all due to 400 BAD REQUEST errors. And the root cause was traced back to a regex check.

This regex was a constraint, where our system only allows people to use English characters to input their first and last names. The issue was, many individuals opted to enter their names in their native languages.

These customers were people who were interested in purchasing health policies from our platform, making them a crucial segment of our user base.

In response to this, we decided to address these 5% of our users by enabling them to enter their names in any language they preferred. But this brought up a lot of challenges that we needed to solve – and I'm going to explain how we did that here.

Challenges with Processing Multilingual Names

1. Data Storage Strategy

We rely on MongoDB for storing and retrieving user names. While MongoDB allows storage for all UTF-8 compatible characters, the problem comes when dealing with search.

For English names, our search operations utilize the simple collation method. The corresponding fields are appropriately indexed to optimize query performance.

While the option to implement a collation index for other languages also exists in MongoDB, this approach means that you have to inform the DB about the specific language for which you intend to search. The challenge here is that our user base spans many languages, with India alone having more than 20 diverse languages.

Our objective was to extend support to at least all Indian languages. But this meant that implementing collation indexes for every supported language would lead to an increased number of indexes – and an increase in index size over time as well.

This approach would also place the responsibility on developers to remember to add an index for each new language as our language support expands, which is far from an efficient solution.

2. API Gateway Constraint

All our APIs are exposed behind an API gateway. Just before the gateway forwards a request to the respective API service, an inbound policy verifies the user's authentication status. Once the user is authenticated, it retrieves basic user details such as name, mobile number, and other metadata, and appends it to a request header of that API.

Many APIs rely on this user-specific data in headers for their further processing.

But there's a restriction imposed by the gateway – it allows only ASCII characters for processing and inclusion in the headers. So we had to make sure that even though the name may be in any other language, the response we shared had to be exclusively in English.

Also, this process needed to remain fast, as any delay in authentication could lead to sluggish API performance.

3. External Partners' Challenge with Vernacular Names

Even if we started to accept names in multiple languages, there were our partners who had to accept those names from us. If they don't support multilingual names, the user journey breaks.

One such example was our payment partner. We had to make sure our payments team always received the English name, even when users provided names in other languages.

Also, we wanted to avoid those annoying pop-ups prompting users to enter their names in English whenever we were able. So keeping these problems in mind, we had to build a viable solution.

How We Solved These Challenges

While utilizing a third-party transliteration service might have been the easiest route, we opted to develop an in-house solution to control costs and maintain full control.

Considering the API gateway and the requirements of payment partners, it became clear that we needed to transform non-English names into English equivalents. But presenting this English name to the user was counterintuitive – for example entering a name in Hindi, only to see it transformed into English upon logging in seemed contradictory.

To handle this, we developed a dual-naming strategy. The original fields, "firstName" and "lastName" would retain the user-entered names in their entered language. Then we introduced two additional fields, "englishFirstName" and "englishLastName" that were dedicated to storing the English counterparts of these names. These English names could then be shared with the API gateway and our payment partners.

Coming back to the challenge of storing these names efficiently, we anticipated that managing collation indexes as the number of supported languages grew would become unmanageable. Searching would also require specifying the collation for each query, creating an added layer of complexity. So we decided to pivot away from this approach.

Our second approach involved using Unicode. As we aimed to support multiple languages without constraints, we recognized that Unicode could effectively represent characters in nearly every language. Because of this, we decided to store Unicode representations for first and last names in their respective MongoDB fields.

We just added another layer between our DB and the application. It converts these Unicode strings to the original values in the local language while retrieving the names from the DB and converting the local names to their respective English names. Then it stores them in englishFirstName and englishLastName at the time of any insert or update.

This strategy provided the flexibility we needed to manage multilingual names seamlessly.

Key Design Considerations

1. Unicode Optimization

Unicode representation typically comprises a 6 character string, with 'a' represented as 'U+0061' and 'P' as 'U+0050,' commonly commencing with 'U+00.' To conserve space in our database storage, we opted to omit the 'U+' prefix and leading zeros, optimizing our data storage.

2. Transliteration vs. Translation

Initially, our aim was transliteration, which requires converting names from one script to another while retaining their phonetic sound.

For example, the Hindi word "प्रतीक्षा" should be transformed to "Partiksha" and not translated to its English equivalent, "Wait".

But we recognized that Google Translate primarily focuses on translation, not transliteration. Again we didn't want to go directly for the paid Google transliteration service in our first iteration, so we developed our transliteration service using the free version of Google translate.

3. Contextual Enhancements

Another and the most crucial observation that we had was providing context to the Google Translate API that influenced its responses.

To leverage this, we experimented with adding statement prefixes to non-English names to establish context. After a few hits and trials, we realized that for shorter names (less than 5 characters), a more extensive prefix statement didn't yield desirable results, and Google often returned the same Hindi word. For longer names, we employed lengthier statements, determining the optimal balance through trial and error.

Translating names normally led to their literal translation. For example "प्रतीक्षा" to "Wait" instead of "Pratiksha":

Adding a prefix statement corrected it:

Alright, now let's see how we actually implemented all this.

Initial Code

After our first iteration, we developed the below code for transliteration. Here we are using the @iamtraction/google-translate library which is a wrapper written over the free Google translate API.

const translate = require('@iamtraction/google-translate');

function getGoogleTranslateText(localName) {
  /*
    Adding an English sentence before the name so that
    it doesn't get translated to its literal meaning.
    For eg परीक्षा to Exam instead of Pariksha.
  */
  if (localName.length <= 5) {
    return `name: ${localName}`;
  }
  return `your name is: ${localName}`;
}

async function translateNameToEnglish(localName) {
  if (localName.match(/^[a-zA-Z ]+$/i)) {
    // If the name is already in English just return
    return localName;
  }
  try {
    const res = await translate(getGoogleTranslateText(localName), {
      to: 'en',
    });
    const translatedName = res.text.split(':')[1].trim();
    return translatedName;
  } catch (err) {}
  // In case of error, Return the Unicode string
  return localName;
}

Beta Release and Production Challenges

Once we built this, we released the feature in beta, and approximately 250 users signed up with non-English names within the first few days.

After simply eyeballing some translated texts, we found that the process of converting the name from its local language to Unicode was working perfectly fine and users were able to view their names properly in the application in the language they preferred.

Still, we identified two issues as far as the process of transliteration to English was concerned:

1. Some names were incorrectly transliterated. This occurrence could be attributed to our dependence on Google Translate, a general translation service, rather than a specialized transliteration service.
2. Some names remained unaltered and were not transliterated. These names were returned in the same language as the original one. This meant adding context with prefix sentences before the translation was causing problems for specific names.

This prompted a further investigation which led us to another npm package called "unidecode," which converts Unicode to the original string. While initial tests with unidecode showed accuracy, they also revealed minor spelling discrepancies. In contrast, Google consistently delivered translations with correct spellings. We just needed to find a way of using the best of both worlds.

So we incorporated unidecode into our algorithm as part of our solution.

Improved Solution

Here's what we came up with:

const translate = require('@iamtraction/google-translate');
const unidecode = require('unidecode');
const { isAlmostEqualStrings } = require('./levenshtein');
/**
 *
 * @param {String} localName
 * @description Generates text for Google (shorter statement context for short names) based on localName length
 * @returns {String} returns text to translate
 */
function getGoogleTranslateText(localName) {
  /*
    Adding an English sentence before name so that
    it doesn't get translated to its literal meaning.
    For eg परीक्षा to Exam instead of Pariksha.
  */
  if (localName.length <= 5) {
    return `name: ${localName}`;
  }
  return `your name is: ${localName}`;
}

/**
 *
 * @param {String} localName
 * @description Give an ALMOST transliterated name
 * @returns {String} returns a converted transliterated name from the local language
 */
function transliterate(localName, googleTranslatedName) {
  const decodedName = unidecode(localName);
  if (
    decodedName &&
    Array.from(decodedName)[0]?.toLowerCase() !==
      Array.from(googleTranslatedName)[0]?.toLowerCase() &&
    !isAlmostEqualStrings(decodedName, googleTranslatedName)
  ) {
    return decodedName;
  }
  return googleTranslatedName;
}

/**
 *
 * @param {String} Input non English string
 * @description translates non-english string to English
 * @returns {String} returns translated string
 */
async function translateNameToEnglish(localName) {
  if (!localName || localName.match(/^[a-zA-Z ]+$/i)) {
    // If name is already in English just return
    return localName;
  }
  try {
    const res = await translate(getGoogleTranslateText(localName), {
      to: 'en',
    });
    const translatedName = res.text.split(':')[1].trim();
    return transliterate(localName, translatedName);
  } catch (err) {}
  // In case of error, Return original string
  return localName;
}

After obtaining the translated name, we feed it into the recently introduced transliterate function. Inside this function, our initial step involves extracting the decoded string using the Unidecode library. But then the crux of the matter arises: wow do we determine which result to prioritize – the decoded string or the translated string?

To tackle this, we implemented Levenshtein Distance, an algorithm that calculates the similarity between two strings.

Initially, we check if the first character of the decoded name matches the first character of the translated name. If it doesn't match, then for sure the translated name was incorrect, so we return the decoded name, even though it might contain minor spelling discrepancies, it's better to use that than the incorrect translation.

If it does match then we go for the Levenshtein Distance algorithm.

The Levenshtein distance is a number that tells you how similar two strings are. The higher the number, the more dissimilar the two strings are.

In the implementation, we have a function isAlmostEqualStrings that generates a value from 0 to 1 and returns true if the value is above a certain threshold. In our case, we set the threshold to 0.8

If the Levenshtein distance indicates a match exceeding 80%, we return the translated name. Otherwise, we return the decoded name. This approach ensures that we prioritize accuracy, offering a reliable result based on the established similarity threshold.

This updated algorithm substantially reduced the above mentioned issues. Even though it's not 100% accurate, it solved our 5% cases very well.

Conclusion

The algorithm we developed was entirely in-house and incurred no costs. While investing in a paid solution might have potentially offered better results, wise engineering decisions taken iteratively and a handful of clever hacks played a vital role in both reducing costs and efficiently resolving the specific problem we had.

The complete code for the above implementation along with the Levenshtein Distance algorithm can be found on GitHub (contributions/corrections are welcome).

With this, we come to the end of the article. My DMs are always open if you want to discuss further on any tech topic or if you've got any questions, suggestions, or feedback in general:

• Twitter
• LinkedIn
• GitHub
• Blog

Happy learning!

As a result, a lot of modern sites and apps use the same format to display large numbers in a compact way. For example, displaying 123,000 as 123K.

freeCodeCamp's YouTube and Instagram profiles using compact number format.

You can do this by writing a custom format function, using a third-party library, or, best of all, using a built-in JavaScript API.

You can of course write your own formatter function (and there are several available on Stack Overflow) but you will end up having to check for a lot of conditions.

As an aside, since ES2021, JavaScript supports using underscores as numeric separators to make large numbers easier to read in your code.

function formatCompactNumber(number) {
  if (number < 1000) {
    return number;
  } else if (number >= 1000 && number < 1_000_000) {
    return (number / 1000).toFixed(1) + "K";
  } else if (number >= 1_000_000 && number < 1_000_000_000) {
    return (number / 1_000_000).toFixed(1) + "M";
  } else if (number >= 1_000_000_000 && number < 1_000_000_000_000) {
    return (number / 1_000_000_000).toFixed(1) + "B";
  } else if (number >= 1_000_000_000_000 && number < 1_000_000_000_000_000) {
    return (number / 1_000_000_000_000).toFixed(1) + "T";
  }
}

formatCompactNumber(12_000);        // 12.0K
formatCompactNumber(2_000_000);     // 2.0M
formatCompactNumber(2_500_000);     // 2.5M
formatCompactNumber(6_000_000_000); // 6.0B
formatCompactNumber(6_900_000_000); // 6.9B

This implementation still leaves a .0 after an even thousand, million, billion, or trillion. You could fix this using the replace method and a regular expression.

function formatCompactNumber(number) {
  if (number < 1000) {
    return number;
  } else if (number >= 1000 && number < 1_000_000) {
    return (number / 1000).toFixed(1).replace(/\.0$/, "") + "K";
  } else if (number >= 1_000_000 && number < 1_000_000_000) {
    return (number / 1_000_000).toFixed(1).replace(/\.0$/, "") + "M";
  } else if (number >= 1_000_000_000 && number < 1_000_000_000_000) {
    return (number / 1_000_000_000).toFixed(1).replace(/\.0$/, "") + "B";
  } else if (number >= 1_000_000_000_000 && number < 1_000_000_000_000_000) {
    return (number / 1_000_000_000_000).toFixed(1).replace(/\.0$/, "") + "T";
  }
}

But what if you need to handle negative numbers? You could add another conditional.

function formatCompactNumber(number) {
  if (number < 0) {
    return "-" + formatCompactNumber(-1 * number);
  }
  if (number < 1000) {
    return number;
  } else if (number >= 1000 && number < 1_000_000) {
    return (number / 1000).toFixed(1).replace(/\.0$/, "") + "K";
  } else if (number >= 1_000_000 && number < 1_000_000_000) {
    return (number / 1_000_000).toFixed(1).replace(/\.0$/, "") + "M";
  } else if (number >= 1_000_000_000 && number < 1_000_000_000_000) {
    return (number / 1_000_000_000).toFixed(1).replace(/\.0$/, "") + "B";
  } else if (number >= 1_000_000_000_000 && number < 1_000_000_000_000_000) {
    return (number / 1_000_000_000_000).toFixed(1).replace(/\.0$/, "") + "T";
  }
}

As you can probably see by now, this only scratches the surface of things you would need to consider when writing your own function for displaying compact numbers.

There are a handful of npm packages for formatting numbers compactly. For example, you could install cldr-compact-number, but this would also add 3 kilobytes (or 1.2 kilobytes gzipped) to your JavaScript bundle, adding slightly to your pageload time.

Thankfully, you don't need to use any third-party libraries to format compact numbers, since there is a relatively simple solution that is natively supported in JavaScript.

How to Use the JavaScript Internationalization API

The JavaScript Internationalization API helps you to support different languages and formatting conventions when working in JavaScript. This could be anything from formatting dates and times to knowing whether a or ä comes first when comparing strings.

The API has excellent browser support, with 98% support worldwide. It works by using the Intl object to create a namespace for language-sensitive string comparison, number formatting, and date and time formatting.

As well as Intl.DateTimeFormat for formatting dates and times, and Intl.Collator for doing language-sensitive string comparison, there is Intl.NumberFormat. This lets you format numbers in a language-specific manner.

To get started, create a formatter using the Intl.NumberFormat constructor. Optionally, you can pass it one or more locales and an options object.

new Intl.NumberFormat(locales, options);

A locale is a parameter that defines the user's language, region, or other localization preference. In this case it's an IETF language tag, such as "en-US" for US English, "zh-CH" for Mandarin, or "uk" for Ukrainian.

In the browser, you can access the user's locale using navigator.language. If you don't provide a locale, the API will attempt to use the user's locale from their browser.

The options object can contain values to control things like how to format currency, whether to use l or liters, or whether to display + or - for positive or negative values.

Once you have created a formatter, you can call its format method and pass it the number you want to format. It will return the number formatted according to whatever configuration you provided.

For example, you could use the Internationalization API to format financial values according to different currencies:

const number = 12345678.99

const germanCurrencyFormatter = new Intl.NumberFormat("de-DE", { style: "currency", currency: "EUR" });

const chineseCurrencyFormatter = new Intl.NumberFormat("zh-CH", { style: "currency", currency: "CNY" });

germanCurrencyFormatter.format(number); // 12.345.678,99 €
chineseCurrencyFormatter.format(number); // ¥12,345,678.99

One of the options that is available is notation. This controls the formatting when displaying numbers. The possible values for notation are:

• "standard" – plain number formatting according to the conventions of the locale (the default)

• "scientific" – returns the order of magnitude for a number

• "engineering" – returns the exponent of ten when the number is divisible by three

• "compact" – returns a string representing the exponent, such as K for thousands

So, for example, if you format the number 123456789 with the locale set to "en", you will get:

• standard: 123,456,789

• scientific: 1.235E8

• engineering: 123.457E6

• compact: 123M

If you want a formatter to display large numbers in a compact manner, set the locale to your desired locale and set notation to "compact" .

const formatter = Intl.NumberFormat("en", { notation: "compact" });

This can be used to create a much shorter formatting function:

function formatCompactNumber(number) {
  const formatter = Intl.NumberFormat("en", { notation: "compact" });
  return formatter.format(number);
}

formatCompactNumber(-57);               // -57
formatCompactNumber(999);               // 999
formatCompactNumber(8_554);             // 8.5K
formatCompactNumber(150_000);           // 150K
formatCompactNumber(3_237_512);         // 3.2M
formatCompactNumber(9_782_716_897);     // 9.8B
formatCompactNumber(7_899_693_036_970); // 7.9T

Now your site or app can display even the largest numbers compactly, making your layout that little bit neater and tidier.

Thanks for reading!

I hope this quick guide helps you when you're working with large numbers in JavaScript and perhaps encourages you to explore the Internationalization API further.

In Japan, I have seen many comments, tweets, and blog posts saying "freeCodeCamp looks awesome, but unfortunately, I can't use it because I can't read English."

Currently, most information and learning resources related to programming are in English. But we want freeCodeCamp to be a place where everyone can learn to code in their native language.

I hope the Japanese freeCodeCamp curriculum will be a new option for Japanese speakers who wish to learn to code.

Japanese translation contributors/supporters

The translation effort at freeCodeCamp is supported by many contributors.

Also, the Japanese translation of the entire curriculum, in particular, was made possible thanks to the support of Mr. Kimio Momose and Mr. Thomas Telandro of the nonprofit organization Unbiased Learning and the translation company TMJ Japan Ltd.

I appreciate all the help from everyone who contributed to this translation effort.

Below is a list of contributors who have participated in the Japanese translation so far.

Japanese translation contributors

yukari-n-erb
mmatsumoto1026
miya-start
tetz-akaneya
furidosu
Besshy
Lulkafe
Emi
Riokeh
King Tony
YAMADA Nobuko
徳輝方
ariadaioh

TMJ Japan Ltd.
tomig7
Hiroko.Nagayama
and others

Supported by: Unbiased Learning

Also, thanks to everyone who gave us kind encouragement in the contributor chat, forum, Twitter, and so on.

Frequently Asked Questions

Below are answers to frequently asked questions about the Japanese curriculum.

I'm halfway through the English curriculum. What should I do?

Your progress and certificates are available in all languages. You don't have to go over it in Japanese again. You can just switch to the Japanese curriculum (making sure you're signed in) and continue with it.

You can switch languages at any time from the menu button at the top right corner of the learning platform.

Also, certificates you already earned in the past can be displayed in Japanese if you would like.

I need help with the curriculum. Where should I ask questions?

The freeCodeCamp forum is the best place to ask questions about the freeCodeCamp curriculum and projects.

In addition, there is now Japanese sub-forum too so you can post questions there. Asking questions in Japanese will help other Japanese campers who face the same problem in the future.

Of course, you can also help the Japanese community grow even more by answering questions from other campers.

I found a translation error or I want to propose improvements to a translation.

We appreciate it if you could let us know in our contributor chat or in the comments on Crowdin, our translation platform.

See this forum post for more details.

It's OK even if you are not 100% sure about your suggestion. Please reach out to us, and we can discuss how we can improve it together.

I want to participate in the translation effort.

We welcome new contributors! Feel free to join the contributor chat and say "Hi".

freeCodeCamp is an open source project, and we are constantly improving and updating our curriculum. New courses will be added from time to time.

Because of this, there will always be more translation work to do each time we add or change something, so we would appreciate your continued contributions.

See below for the information regarding Japanese translation contribution:

• Forum post: freeCodeCamp contribution information in Japanese
• Contribution guidelines (Japanese)

You can also contribute to the source code on GitHub.

I can't participate in translation, but I want to help freeCodeCamp and the Japanese community.

There will be more and more things we need help with as the Japanese community grows.

Please reach out to me in the contributors chat. Let's talk about how we can help the Japanese freeCodeCamp community together.

If you want to learn more about freeCodeCamp, you can read other frequently asked questions here.

Today I'm excited to announce that we've launched a Hindi-language YouTube channel.

The freeCodeCamp Hindi YouTube Channel

So far, we already have two full courses on the channel: Flutter and Python.

And Saturday we will live-stream the first segment of our 5-week-long Data Science with Python course.

You can subscribe to the freeCodeCamp Hindi channel here.

This channel is run by prolific YouTuber Pawan Kumar.

Pawan is a Google Developer Expert for technologies like Flutter, Dart, and Firebase. He also runs a tech youtube channel called MTECHVIRAL.

Pawan works as a head of Mobile Engineering at Frontier Wallet. He started his career as an Android dev when he was in college. He likes to experiment with new technologies, and talk about them at events and on his Twitter.

Pawan has helped many developers until now around the world in learning new technologies and getting tech jobs.

"When I was a child, I didn’t have many resources to learn new things. So I decided to help others in learning new awesome things." – Pawan Kumar

And this lead to him creating the official Hindi-language freeCodeCamp YouTube channel.

The future of freeCodeCamp's Internationalization Efforts

Our community is making steady progress in our internationalization efforts. Look for some other big announcements in the coming months.

Our goal is to eventually have a translated curriculum for every major world language, launch YouTube channels in more major languages.

Again, you can subscribe to the Hindi YouTube channel here.

Happy coding.

When you develop some software you may not think about timezones at first. Unless you live in a country which has to deal with multiple time zones, such as the United States or Russia.

I recently came across an issue involving timezones. There were some unit tests making assertions about dates that used to work at my office in France but weren't working in Morocco for new members on our team.

Here is the unit test working in France but not in Morocco

‌This was an opportunity for me to learn how to correctly handle dates and times for international software. In this article, I’ll introduce time zone issues and share some rules to follow.

Quick introduction to time zones

As the earth is kind of a sphere, the sun is rising in Japan while it's setting in America. If everyone used global time, let’s say 09:00 would be sunrise in Japan, but for Americans it would be sunset. Not very handy.

To make sure the time is coordinated with the sun for everyone, it’s necessary to shift from global time according to your location. As a result, the globe gets split into time zones and each gets an offset. This offset is a number of minutes to add to the global time to get your time zone time. It can be either positive or negative.

_Standard world time zones — Illustration by [Wikimedia Commons](https://commons.wikimedia.org/wiki/User:Hellerick" rel="noopener">Hellerick from <a href="https://en.wikipedia.org/wiki/File:Standard_World_TimeZones.png" rel="noopener)

Global time is called UTC, it stands for Coordinated Universal Time. You may also heard about GMT which is a time zone without any offset.

For instance, when it’s 10:50 at UTC, it’s also 03:50 in San Francisco with a -0700 offset and 18:50 in Beijing with a +0800 offset. Yet, the shift isn’t only in whole hours: Nepal's offset is +0545. You can check it out on Wikipedia.

In addition of this offset, which comes with the time zone, some countries also shift clocks twice a year. DST or summer time adds one hour to the time zone offset before summer. Then, the clock is reset to the time zone time in winter. The goal is to make the daytime longer.

The most common way to figure out a time zone is by using the IANA Time Zone Database. You end up with a string such as Europe/Paris following the Area/City pattern. Besides, Microsoft maintains its own Microsoft Time Zone Database used on its operating systems. But this can cause issues when running cross-platform .NET Core apps.

IANA is still the go-to. The Microsoft database isn't updated often, it contains less history, fairly curious time zone names (eg: Romantic Standard Time) and is error prone. For example, try to not mix up Arab , Arabic and Arabian Standard Time. For more details on each database and their differences, check out this article.

One last thing: there are plenty of ways to write a date. Fortunately, the ISO 8601 specification sets a common rule for date formatting.

November 11, 2018 at 12:51:43 AM (in a time zone at UTC+00:00)
2018-11-05T12:51:43Z <- Z stands for UTC

November 11, 2018 at 12:51:43 AM (in a time zone at UTC +07:30)
2018-11-05T12:51:43+0730

How computers handle dates

Computers are only able to perform operations using numbers. This means that 2020-08-01 +1 is not equal to 2020-08-02 and can’t be handled.

In order to work with dates more easily, we can represent dates as numbers. This is what timestamps are all about. It’s the number of milliseconds elapsed from a pre-defined date (or epoch) to the specified date.

Great, let’s choose an epoch then! Actually, the common epoch has already been set and its value is January 1, 1970 (midnight UTC).

To make sure you understood, run the previous snippet in your browser. What? You didn’t get the same result?

Ok, I cheated a bit to get this result… I should get Thu Jan 01 1970 01:00 GMT+0100 because my computer time zone is set to Europe/Paris.

Actually, this moment with a zero timestamp is midnight in Greenwich, but also 05:45 in Mumbai and even 1969-12-31T16:30 in San Francisco when you consider their time zone’s offset.

Rule #1 : Timestamps are only for saving, not for displaying. It's considered on UTC because it doesn’t include any offset or time zone.

You didn’t get the “right” date before because JavaScript uses your local time zone to show the most accurate date/time to you.

Now, try the following snippet. I’m sure you’ll get the same result as I did:

Yes, the zero timestamp is 1970-01-01T00:00:00 at UTC for everyone around the globe. Still, it’s not true if you choose another time zone.

To sum up, toString shows the date using your local time zone while toUTCString is based on UTC. Also don’t be fooled by toISOString which is the same as toUTCString but outputs the ISO 8601 format (its name should be toUTCISOString).

I recommend the date command to convert a second timestamp (not milliseconds) into a readable string. Using this command with the UTC option makes sure it doesn't take your computer/browser's time zone into account.

# Linux
$ date -d @1586159897 -u 
Mon Apr  6 07:58:17 UTC 2020

# For Osx users
$ date -r 1586159897 -u

Let’s fix our unit test

The problem I encountered with time zones was in my unit tests. Take the time to read it and understand what it’s supposed to assert:

In this test, the goal is to check that setHours sets the date’s hours and minutes to zero (midnight). I first choose a random timestamp which isn’t at midnight. Then compare the result with the timestamp for the same day at midnight.

Actually it’s working – but only if your time zone offset is +0200 (including DST) at this moment. For instance, it’s not working for Africa/Casablanca ( +0100 including DST). Let’s see how those timestamps are printed:

That’s it, the UTC date for both results isn't the same. It also means the resulting timestamps aren’t the same either.

As you can see, the offset for Paris is +0200 and +0100 for Casablanca. But both display midnight with toString. This means that the setHours function uses your computer time zone to perform the operation. And toString displays the date using your time zone.

This is not the only issue with this test: what if you run this test in San Fransisco? Right, the day would be 2020-07-31 for both dates because of the -0700 offset.

The safest way to make this test reliable and work all around the world is to use a date in your local time zone. You’ll not use timestamps to set initial dates anymore.

We can enhance the previous rule about timestamps:

Rule #2 : String dates are suitable for display using the user's time zone and computations. They aren’t on UTC but generally include an offset.

Keep it on date on the server side

The rule about timestamps still applies on the server side. However the second rule about using string dates can’t be used.

Indeed, in some case with technologies such as PHP, Java, and Rails the pages are rendered on server side (SSR). This means all the HTML is generated by the server and it has no idea about the client's time zone. Think about the server – it’s nothing more than a computer on the globe. It also has its own time zone but it’s not necessarily the same as the client's time zone.

Rule #3 : Servers might either know the client's time zone or send a date on UTC. The server's time zone doesn’t matter.

The new Java 8 Date/Time is considered one of the most understandable and clear APIs that helps you deal with date. I’m not going to explain how it works here but let’s review some interesting points.

LocalDateTime, OffsetDateTime and ZonedDateTime are the 3 classes provided to compute and display date and time. No more Date or DateTime which mix up displaying the local date and UTC date.

The following examples are extracted from this awesome article (written by Jonas Konrad) which describes the Java 8 Date/Time API with a bunch of examples. By the way, many thanks to him, he kindly let me quote its pieces of code!

Let’s look at the differences between the 3 classes:

There is a small but important difference between OffsetDateTime and ZonedDateTime, did you notice it?

As its name says, OffsetDateTime is only aware of an offset between the local date and UTC. This means that it handles DST differently from a date which is attached to a time zone.

The example with a time zone seems to be the right behavior. Actually, both are correct because adding 1 day can either mean:

• Add 1 day and keep the same hour (handles DST with ZonedDateTime)
• Add 24 hours to the current date (with OffsetDateTime).

Remember Rule #1 about timestamps? You should only use a UTC timestamp for saving. The Java API provides an Instant class which is a timestamp you can get from any of the three classes used for displaying the date.

Final thoughts

In this article, you've learned that timestamps are for saving (Rule #1) and string dates are for displaying (Rule #2). Did you notice that the number of seconds from the epoch is quite a big number?

That’s why after the Unix Millennium Bug (Y2K) problem comes the Y2K38 problem which stands for the year 2038. At 2038-01-19T03:14:07Z the timestamp (in seconds) will reach its maximum for 32-bit signed integers 2,147,483,647 . It will then turn into a negative number after adding one more second.

_This sign indicates January 1900 instead of January 2000 — Picture from [Wikipedia Commons](https://en.wikipedia.org/wiki/File:Bug_de_l%27an2000.jpg" rel="noopener)

‌On forums, people say they don’t care because their software won't be used for 20 years without re-writing. Well, that might be true but let’s still think about some solutions (with MySQL):

• Update TIMESTAMP type to 64-bit signed integers
• Save UTC dates in DATETIME columns instead of TIMESTAMP

Both solutions have their advantages and drawbacks. The first one seems like a hack which reports the issue later. Yet, it fixes the issue for an almost infinite amount of time (billions of years). Your software will be deprecated and not used anymore when the problem occurs again.

The second solution also works for a very long time (up to 9999-12-31T23:59:59Z).

Using TIMESTAMP is recommended for logs, while DATETIME is better for other needs. Remember a timestamp can’t store a date prior to 1970-01-01T00:00:00Z and not after 2038-01-19T03:14:07Z. This means you should use DATETIME to save dates far in the past and future.

Besides, in MySQL TIMESTAMPs are stored at UTC but displayed according to a specified time zone (and converted to UTC before saving). This mechanism comes in handy when you need to get a local date and doesn’t exist with DATETIME.

A last word about moment.js, a popular library to deal with dates. I first experimented an issue and wanted to warn you about it:

Both console.logs will output 2020-08-02 00:00. If you’re used to functional programming, you expect hours and minutes to return a new moment object because they are pure functions. It’s not the case – they modify the input date and return it for easy chaining.

Thanks for reading up to the end. I hope this experience of mine has been useful to you. By the way, I’m not very confident about the choice between TIMESTAMP and DATETIME, so don’t hesitate to share your experience!

If you found this article useful, please share it on social media to help others find it and to show your support! ?

Don’t forget to check my author page for upcoming articles ?

In this article you are going to learn how to translate your Rails application into multiple languages, work with translations, localize datetime, and switch locales. We are going to see all these aspects in action by creating a sample application and enhancing it step by step. By the end of the article you will have all the necessary knowledge to start implementing these concepts in real projects.

Preparing your Rails App

So, as I already said, we are going to see all the concepts in action, therefore let’s create a new Rails application by running:

rails new SampleApp

For this tutorial I am using Rails 5.2.1, but most of the described concepts apply to older versions as well.

Now let’s generate a StaticPagesController which is going to have an index action (our main page):

rails g controller StaticPages index

Tweak the views/static_pages/index.html.erb view by adding some sample content:

<h1>Welcome!</h1> <p>We provide some fancy services to <em>good people</em>.</p>

Also I would like to add a Feedback page where our users will be able to share their opinion (hopefully, a positive one) about the company. Each feedback will have an author’s name and the actual message:

rails g scaffold Feedback author message

We will be interested only in two actions: new (which is going to render the form to post a review and also list all the existing reviews) and create (to actually validate and persist the reviews). Of course, ideally the reviews should be pre-moderated but we won’t bother with this today.

Tweak the new action to fetch all the reviews from the database and order them by creation date:

# feedbacks_controller.rb # ... def new @feedback = Feedback.new @feedbacks = Feedback.order created_at: :desc end

Also I would like to redirect the user to the Feedback page when the form is processed and the new record is persisted:

# feedbacks_controller.rb # ... def create @feedback = Feedback.new(feedback_params) if @feedback.save redirect_to new_feedback_path else @feedbacks = Feedback.order created_at: :desc render :new end end

Render the feedbacks collection on the new page:

<!-- views/feedbacks/new.html.erb --> <!-- other code goes here... --> <%= render @feedbacks %>

Lastly, create a partial for an individual feedback:

<!-- views/feedbacks/_feedback.html.erb --> <article> <em> <%= tag.time feedback.created_at, datetime: feedback.created_at %><br> Posted by <%= feedback.author %> </em> <p> <%= feedback.message %> </p> <hr> </article>

Take care of the routes:

# config/routes.rb Rails.application.routes.draw do resources :feedbacks root 'static_pages#index' end

Lastly add a global menu to the layout:

<!-- views/layouts/application.html.erb --> <!-- other code goes here... --> <nav> <ul> <li><%= link_to 'Home', root_path %></li> <li><%= link_to 'Feedback', new_feedback_path %></li> </ul> </nav>

Now run migrations and boot up the server:

rails db:migrate rails s

Navigate to the http://locahost:3000 and make sure that everything is fine. Now that we have something to work with, let’s proceed to the main part and localize our application.

A Bit of Configuration

Before performing translations, we need to decide which languages will be supported. You can choose any, but I will stick with Russian and English, with the latter set as a default. Reflect this inside the config/application.rb file:

# ... config.i18n.available_locales = [:en, :ru] config.i18n.default_locale = :en

Also hook up a rails-i18n gem that has locale data for different languages. For example, it has translated names of the months, pluralization rules, and other useful stuff.

# Gemfile # ... gem 'rails-i18n'

Just install this gem and you are good to go:

bundle install

Storing Translations

Now that everything is configured, let’s take care of the home page and translate the text there.

The simplest way to do this is by utilizing localized views. All you need to do is create views named index.LANG_CODE.html.erb, where the LANG_CODE corresponds to one of the supported languages. So, in this demo we should created two views: index.en.html.erb and index.ru.html.erb. Inside just place content for English and Russian version of the site, and Rails will automatically pick the proper view based on the currently set locale. Convenient, eh?

This approach, however, is not always feasible. Another way would be to store your translated strings in a separate file, and render a proper version of the string based on the chosen language. By default, Rails employs YAML files that has to be stored under the config/locales directory. Translations for different languages are stored in separate files, and each file is named after this language.

Open the config/locales folder and note that there is already an en.yml file inside which has some sample data:

en: hello: "Hello world"

So, en is a top-level key representing the language that these translations are for. Next, there is a nested key-value pair, where hello is the translation key, and Hello world is the actual translated string. Let’s replace this pair with the following content:

en: welcome: "Welcome!"

This is just a welcoming message from our homepage. Now create a ru.yml file in the config/locales folder and provide translated welcoming message there as well:

ru: welcome: "Добро пожаловать!"

We have just created translation for our first string, which is really great.

Performing Simple Translations

Now that we have populated the YAML files with some data, let’s see how to employ the translated strings in the views. Actually, it is as simple as utilizing the translate method which is aliased as t. This method has one required argument: the name of the translation key:

<!-- views/static_pages/index.html.erb --> <h1><%= t 'welcome' %></h1>

When the page is requested, Rails looks up the string that corresponds to the provided key, and renders it. If the requested translation cannot be found, Rails will just render the key on the screen (and turn it to a more human-readable form).

Translation keys can be named anything you like (well, nearly anything) but of course it is advised to give them some meaningful names so that you can understand what text they correspond to.

Let’s take care of the second message:

en: welcome: "Welcome!" services_html: "We provide some fancy services to <em>good people</em>."

ru: welcome: "Добро пожаловать!" services_html: "Мы предоставляем различные услуги для <em>хороших людей</em>."

Why do we need this _html postfix? Well, as you can see our string has some HTML markup, and by default Rails will render the em tag as plain text. As long as we don’t want this to happen, we mark the string as a “safe HTML”.

Now just use the t method again:

<!-- views/static_pages/index.html.erb --> <!-- ... ---> <p><%= t 'services_html' %></p>

More On Translation Keys

Our homepage is now localized, but let’s stop for a moment and think about what we have done. All in all, our translation keys have meaningful names, but what happens if we are going to have, say, 500 messages in the app? This number is actually not that big, and large websites may have thousands of translations.

If all our key-values pairs are stored right under the en (or ru) key without any further grouping, this leads to two main problems:

• We need to make sure that all the keys have unique names. This becomes increasingly complex as your application grows.
• It is hard to locate all related translations (for example, translations for a single page or feature).

Therefore, it would be a good idea to further group your translations under arbitrary keys. For example, you may do something like this:

en: main_page: header: welcome: "Welcoming message goes here"

The level of nesting is not limited (but you should be reasonable about it), and the keys in different groups may have identical names.

It is beneficial, however, to follow the folder structure of your views (in a moment we will see why). Therefore, tweak the YAML files in the following way:

en: static_pages: index: welcome: "Welcome!" services_html: "We provide some fancy services to <em>good people</em>."

ru: static_pages: index: welcome: "Добро пожаловать!" services_html: "Мы предоставляем различные услуги для <em>хороших людей</em>."

Generally, you need to provide full path to the translation key when referencing it in the t method:

<!-- views/static_pages/index.html.erb --> <h1><%= t 'static_pages.index.welcome' %></h1> <p><%= t 'static_pages.index.services_html' %></p>

However, there is also a “lazy” lookup available. If you perform translation in a view or controller, and the translation keys are namespaced properly following the folder structure, you may omit the namespaces all together. This way, the above code turns to:

<!-- views/static_pages/index.html.erb --> <h1><%= t '.welcome' %></h1> <p><%= t '.services_html' %></p>

Note that the leading dot is required here.

Let’s also translate our global menu and namespace the translations properly:

en: global: menu: home: "Home" feedback: "Feedback"

ru: global: menu: home: "Главная" feedback: "Отзывы"

In this case we can’t take advantage of the lazy lookup, so provide the full path:

<!-- views/layouts/application.html.erb --> <!-- ... ---> <nav> <ul> <li><%= link_to t('global.menu.home'), root_path %></li> <li><%= link_to t('global.menu.feedback'), new_feedback_path %></li> </ul> </nav>

Translating Models

Now let’s proceed to the Feedback page and take care of the form. The first thing we need to translate is the labels for the inputs. It appears that Rails allows us to provide translations for the model attributes, and they will be automatically utilized as needed. All you need to do is namespace these translations properly:

en: activerecord: attributes: feedback: author: "Your name" message: "Message"

ru: activerecord: attributes: feedback: author: "Ваше имя" message: "Сообщение"

The labels will now be translated automatically. As for the “submit” button, you can provide translation for model itself by saying:

en: activerecord: models: feedback: "Feedback"

But honestly I don’t like the “Create Feedback” text on this button, so let’s stick with a generic “Submit” word:

en: global: forms: submit: Submit

ru: global: forms: submit: Отправить

Now utilize this translation:

<!-- views/feedbacks/_form.html.erb --> <!-- ... ---> <%= form.submit t('global.forms.submit') %>

Error Messages

Probably we do not want the visitors to post empty feedback messages, therefore provide some simple validation rules:

# models/feedback.rb # ... validates :author, presence: true validates :message, presence: true, length: {minimum: 5}

But what about the corresponding error messages? How do we translate them? It appears that we don’t need to do anything at all as rails-i18n gem already knows how to localize common errors. For example, this file contains error messages for the Russian locale. If you actually do want to tweak the default error messages, then check the official doc that explains how to achieve that.

One problem with the form, however, is that the error messages subtitle (the one that says “N errors prohibited this feedback from being saved:”) is not translated. Let’s fix it now and also talk about pluralization.

Pluralization Rules

As long as potentially there can be one or more error messages, the “error” word in the subtitle should be pluralized accordingly. In English words are usually pluralized by adding an “s” postfix, but for Russian the rules are a bit more complex.

I already mentioned that the rails-i18n gem contains pluralization rules for all the supported languages, so we don’t need to bother writing them from scratch. All you need to do is provide the proper key for each possible case. So, for English there are only two possible cases: one error or many errors (of course, there can be no errors, but in this case the message won’t be displayed at all).

en: global: forms: submit: Submit messages: errors: one: "One error prohibited this feedback from being saved" other: "%{count} errors prohibited this feedback from being saved"

The %{count} here is interpolation – we take the passed value and place it right into the string.

Now take care of the Russian locale which has more possible cases:

ru: global: forms: submit: Отправить messages: errors: one: "Не удалось сохранить отзыв! Найдена одна ошибка:" few: "Не удалось сохранить отзыв! Найдены %{count} ошибки:" many: "Не удалось сохранить отзыв! Найдено %{count} ошибок:" other: "Не удалось сохранить отзыв! Найдена %{count} ошибка:"

Having this in place, just utilize these translation:

<!-- views/feedbacks/_form.html.erb --> <!-- ... ---> <%= form_with(model: feedback, local: true) do |form| %> <% if feedback.errors.any? %> <div id="error_explanation"> <h2><%= t 'global.forms.messages.errors', count: feedback.errors.count %></h2> <!-- errors... --> </ul> </div> <% end %> <!-- form fields --> <% end %>

Note that in this case we pass the translation key as well as the value for the count variable. Rails will take the proper translation variant based on this number. Also the value of the count will be interpolated into each %{count} placeholder.

Our next stop is the _feedback.html.erb partial. Here we need to localize two strings: “Posted by…” and datetime (created_at field). As for “Posted by…”, let’s just utilize the interpolation again:

en: global: feedback: posted_by: "Posted by %{author}"

ru: global: feedback: posted_by: "Автор: %{author}"

<!-- views/feedbacks/_feedback.html.erb --> <article> <em> <%= tag.time feedback.created_at, datetime: feedback.created_at %><br> <%= t 'global.feedback.posted_by', author: feedback.author %> </em> <p> <%= feedback.message %> </p> <hr> </article>

But what about the created_at? To take care of it, we can take advantage of the localize method aliased as just l. It is very similar to the Ruby’s strftime, but produces a translated version of the date (specifically, the months’ names are translated properly). Let’s use a predefined format called :long:

<!-- views/feedbacks/_feedback.html.erb --> <article> <em> <%= tag.time l(feedback.created_at, format: :long), datetime: feedback.created_at %><br> <%= t 'global.feedback.posted_by', author: feedback.author %> </em> <!--... --> </article>

If you would like to add your very own format, it is possible too as explained here.

Switching Between Locales

So, our app is now fully translated… but there is a very minor thing: we cannot change the locale! Come to think of it, this is quite a major issue really, so let’s fix it now.

There are a handful of possible ways of setting and persisting the chosen locale across the requests. We are going to stick with the following approach:

• Our URLs will have an optional :locale parameter, and so they’ll look like [http://localhost:3000/en/some_page](http://localhost:3000/en/some_page)
• If this parameter is set and the specified locale is supported, we translate the app into the corresponding language
• If this parameter is not set or the locale is not supported, set a default locale

Sounds straightforward? Then let’s dive into the code!

First of all, tweak the routes.rb by including a scope:

# config/routes.rb scope "(:locale)", locale: /#{I18n.available_locales.join("|")}/ do # your routes here... end

Here we are validating the specified parameter using a RegEx to make sure that the locale is supported (note that the anchor characters like \A are not permitted here).

Next, set a before_action in the ApplicationController to check and set the locale on each request:

# application_controller.rb # ... before_action :set_locale private def set_locale I18n.locale = extract_locale || I18n.default_locale end def extract_locale parsed_locale = params[:locale] I18n.available_locales.map(&:to_s).include?(parsed_locale) ? parsed_locale : nil end

Also, in order to persist the chosen locale across the requests, set the default_url_options:

# application_controller.rb # ... private def default_url_options { locale: I18n.locale } end

The is going to include the locale parameter into every link generated with Rails helpers.

The last step is to present two links to switch between locales:

<!-- views/layouts/application.html.erb --> <!-- ... --> <nav> <ul> <li><%= link_to t('global.menu.home'), root_path %></li> <li><%= link_to t('global.menu.feedback'), new_feedback_path %></li> </ul> <ul> <li><%= link_to 'English', root_path(locale: :en) %></li> <li><%= link_to 'Русский', root_path(locale: :ru) %></li> </ul> </nav>

As an exercise, you may make these links more fancy and, for instance, redirect the user back to the page that he was browsing.

Simplify Your Life With Lokalise

By now you are probably thinking that supporting multiple languages on a big website is probably a pain. And, honestly, you are right. Of course, the translations can be namespaced, and even split into multiple YAML files if needed, but still you must make sure that all the keys are translated for each and every locale.

Luckily, there is a solution to this problem: the Lokalise platform that makes working with the localization files much simpler. Let me guide you through the initial setup which is nothing complex really.

• To get started, grab your free trial
• Install Lokalise CLI that will be used to upload and download translation files
• Open your personal profile page, navigate to the “API tokens” section, and generate a read/write token
• Create a new project, give it some name, and set English as a base language
• On the project page click the “More” button and choose “Settings”. On this page you should see the project ID
• Now from the command line simply run lokalise --token <token> import <project_id> --lang_iso en --file config/locales/en.yml while providing your generated token and project ID (on Windows you may also need to provide the full path to the file). This should upload English translation to Lokalise. Run the same command for the Russian locale.
• Navigate back to the project overview page. You should see all your translation keys and values there. Of course, it is possible to edit, delete them, as well as add new ones. Here you may also filter the keys and, for example, find the untraslated ones which is really convenient.
• After you are done editing the translations, download them back by running lokalise --token <token> export <project_id> --type yaml --bundle_structure %LANG_ISO%.yml --unzip_to E:/Supreme/docs/work/lokalise/rails/SampleApp/config/locales/. Great!

Lokalise has many more features including support for dozens of platforms and formats, ability to order translations from professionals, and even the possibility to upload screenshots in order to read texts from them. So, stick with Lokalise and make your life easier!

Conclusion

In this article we have thoroughly discussed how to introduce internationalization support in Rails applications and implemented it ourselves. You have learned how and where to store translations, how to look them up, what are localized views, how to translate error messages and ActiveRecord-related stuff, as well as how to switch between locales and persist the chosen locale among the request. Not bad for today, eh?

Of course, it is impossible to cover all ins and outs of Rails I18n in one article, and so I recommend checking out the official guide that gives some more detailed information on the topic and provides useful examples.

Originally published at blog.lokalise.co on August 23, 2018.