Strange encoding after production build of SAPUI5 app

[julian-s]

Hello,

We are currently facing a problem with the WebIDE deployment process to our own SAPUI5 ABAP Repository.

We have a prepared an i18n.properties file, where all the German strings are saved.

If we start the application on the WebIDE itself, then the encoding of the strings works perfectly as expected. But when we are now trying to build and deploy the app on the ABAP repository, it will fail with the encoding. During the build process on the WebIDE, it convert the the i18n properties file in another encoding. We see the file here from the dist folder.

As a result, we see a wrong encoded application:

Now the question is, what can we do to set the correct charset and get the expected result.

Thanks in advance.

[boghyon]

Could you share package.json and ui5.yaml with us? Seems like a common issue UI5 devs encounter nowadays. One of the solutions would be to tell the tooling to read the *.properties files with UTF-8 when converting. See https://stackoverflow.com/a/58835799/5846045

[abdulbasit · Accepted solution]

I had the similar problem and thanks to boghyon.hoffmann, he suggested me the solution on OpenUI5 Slack channel. (https://openui5.slack.com/archives/C61AA31MH/p1572532524039400?thread_ts=1572529459.031600&cid=C61AA31MH).

You need to set the propertiesFileSourceEncoding parameter in ui5.yaml file:

resources:
    configuration:
        propertiesFileSourceEncoding: UTF-8

I hope this can fix your problem.

[boghyon · Accepted solution]

Thanks for the mention. 🙂 Just a small side note for other readers: the `propertiesFileSourceEncoding` is supported only as of @ui5/cli 1.7.0+

So in the package.json:

"devDependencies": {
  "@ui5/cli": "<at least 1.7.0>",
  ...
},

___

The latest version can be found here: https://github.com/SAP/ui5-cli/releases
Or with the command: npm show @ui5/cli version

[julian-s · Accepted solution]

Hello everyone,

Thanks for the answers and comments. So I have added the three lines from 1bf660afcf81417ca60d42962287a506 in my .yml file.

It looks like this now, but unfortunately after the build, the properties file still looks the same.

I also have checked the package.json file as boghyon.hoffmann mentioned. The version matches with your suggestion.

{
  "name": "zmmwe",
  "version": "0.0.1",
  "description": "",
  "private": true,
  "devDependencies": {
    "@ui5/cli": "1.7.0",
    "@sap/ui5-builder-webide-extension": "1.0.5"
  },
  "scripts": {
    "build": "ui5 build --include-task=generateManifestBundle generateCachebusterInfo"
  },
  "ui5": {
    "dependencies": [
      "@sap/ui5-builder-webide-extension"
    ]
  }
}

[abdulbasit · Accepted solution]

It'll again convert the encoding but this time the new conversion should be correct. Did you test the result on the browser?

[Sschlegel · Accepted solution]

Hi,

thanks for the hint. To make it work for me, I need to add to the ui5-deploy.yaml - working with VSCode.

Regards

Sören

[julian-s]

I have build and deployed it now again and suddendly the property files are now showing up correctly. Maybe the WebIDE showed up an older version of the dist properties file. It works now, thank you very much.

Update: But I have added an additional thing in the package.json. When I add

--exclude-task escapeNonAsciiCharacters

to the build tag, then it will display the characters also in the dist folder correctly and in the end on the productive system.

So my package.json looks now like this:

{
  "name": "zmmwe",
  "version": "0.0.1",
  "description": "",
  "private": true,
  "devDependencies": {
    "@ui5/cli": "1.7.0",
    "@sap/ui5-builder-webide-extension": "1.0.5"
  },
  "scripts": {
    "build": "ui5 build --include-task=generateManifestBundle --exclude-task escapeNonAsciiCharacters generateCachebusterInfo"
  },
  "ui5": {
    "dependencies": [
      "@sap/ui5-builder-webide-extension"
    ]
  }
}

[boghyon]

Hi Julian,

The --exclude-task escapeNonAsciiCharacters command could be a workaround but we still believe that it shouldn't be necessary.

Without that command, if you build with propertiesFileSourceEncoding: UTF-8, do you still see Bez/u00c3/u00bcger (Bezüger)? Or is it Bez\u00fcger (Bezüger) in the *.properties file?

[julian-s]

Hi boghyon.hoffmann,

No I still saw the Bez/u00c3/u00bcger (Bezüger) when I just updated the .yml file.

[gregorw]

Please be aware that according to Java-Properties-Datei the properties file is ISO 8859-1 encoded and not UTF-8.

[pieterjanssens]

In Java SE 9, properties files are loaded in UTF-8 encoding. In previous releases, ISO-8859-1 encoding was used for loading property resource bundles. UTF-8 is a much more convenient way to represent non-Latin characters.

https://docs.oracle.com/javase/9/intl/internationalization-enhancements-jdk-9.htm#JSINT-GUID-974CF48...

[AndreasKunz]

Hi, I can't fully answer, but as the converted Unicode character code \u00c3 is already wrong (the "A" with tilde), the problem is definitely during the conversion. Seems like this conversion assumes the wrong charset for the input file. Or the input file HAS a wrong encoding. What is the encoding? You can change it with tools like Notepad++, maybe switching to or from UTF-8 helps.

[julian-s]

Hi Andreas,

thank you for your answer. I have exported the file into Notepad++, but it shows me that it is already encoded in UTF-8. Also the characters (especially the umlauts) are showing up correctly, as you can see in the picture.

I have already tried to convert the document to UTF-8-BOM, but ending in the build process with the same result.