Problems with german characters (de_DE.utf8) using trac importer

Question

Hello,

I am trying to import a companies trac project into jira. Currently I have a problem when importing issues containing special german chars. It seems that all umlauts occurring in the trac ticket description are not correctly recognized. After the import they show up as questionmarks (?).

I made sure that all 'layers' and components are configured to us utf8:

* the trac db,
* the os running trac
* jira-app
* jira-db (postgres)
* both os running the db and the app

Actually it somehow works: Umlauts used in the trac ticket summary are correctly handled during import. No problem here. Also when creating new issues in jira: no problem here ! Importing issues with umlauts using csv importer works as well. No problems with umlauts here !

I checked the postgres db: The questionmarks occur already in the jiraissue table (column description). I noticed that the type postgres db type is VARCHAR rsp. TEXT for column summary rsp. description. However, I dont believe that this matters.

Any ideas ?

Internationalization settings:

Indexing languageEnglish - Aggressive StemmingInstalled languagesEnglish (UK)
 English (United States)
 French (France)
 German (Germany)
 Japanese (Japan)
 Spanish (Spain)Default languageEnglish (United States)Default user time zone(GMT-06:00) GMT-6

some more settings:

Java Version1.7.0Java VendorOracle CorporationJVM Version1.7JVM VendorOracle CorporationJVM Implementation Version21.0-b17Java RuntimeJava(TM) SE Runtime EnvironmentJava VMJava HotSpot(TM) 64-Bit Server VMUser NamejiraUser TimezoneEurope/BerlinUser LocaleEnglish (United States)System EncodingANSI_X3.4-1968Operating SystemLinux 3.13.0-77-genericOS Architectureamd64Application Server ContainerApache Tomcat/7.0.55Database typepostgres72Database JNDI addresspostgres72 jdbc:postgresql://172.17.0.3:5432/postgresDatabase URLjdbc:postgresql://172.17.0.3:5432/postgresDatabase version9.4.0Database driverPostgreSQL Native Driver PostgreSQL 9.0 JDBC4 (build 801)

JIRA info:

Version6.3.15Build Number6346Build DateTue Feb 03 00:00:00 CET 2015Build Revisiondbc023dd75cecacf443c4b235f66124b15f5c5fe

thiema · Answer

Hello Moe,

I should have mentioned, that I am using the "Trac Importer" (https://confluence.atlassian.com/jira/importing-data-from-trac-238617182.html). Hence there is no csv or excel data. I directly import the zipped trac.db and other files, as described in the reference.

Regarding csv imports in general: I tried to import a test csv file and found there is no problem with character encoding. However, currently I think using csv import method is no good option when importing the entire project from trac with thousands of issues.

Mohamad Khalife · Answer

The cause of this problem is the difference between the encoding in the Excel file and CSV. The following steps shows how to solve it:

* Have a clean export Excel file ready to be imported
* Make sure the file doesn't contain any commas
* Save the file as Unicode Text (*.txt)
* Now you will have a text file with data separated by tabs
* Open the file in Notepad
* Select the space between two different values and copy it
* Click ctrl+H to open the replace toolbox
* Paste what you have selected in the Find what text field
* Type "," in the Replace with text field
* Click Replace All
* Copy all data from that file.
* Create a new text file and paste the copied data.
* Save the text file with Unicode encoding.
* Change its extension to CSV.
* Now if you open the saved CSV file, you will see that it is not delimited based on ",". To solve it:* From the same opened CSV file, click on "Data"
* Click on "From Text"and Browse for the new saved CSV file and click on "Open"
* Choose "Delimited", Next"
* Uncheck all values except for "Comma" and make sure that "Treat consecutive delimiters as one" is unchecked
* Click on "Next", "Finish" and "Ok"

* Now you will have your CSV file ready for import