[CJK] Unity dash application search returns irrelevant applications but all relevant applications are shown

Bug report courtesy of <email address hidden>

This may either be a legitimate bug in the matching code, or be a bug in the metadata of the Sudoku application itself.

Brandon: can you try to reproduce and see why the match actually happens?

Additional test result:

Searching for “语” in the 'dash' shows 2 relevant applications and 2 irrelevant applications. “ 语” appears in “Language Support” and “Region and Language” name in Chinese (simplified) but not in “gbrainy” or “Orca”.

I was looking at this from another bug (forgot which one) and I was getting the same results. Zeitgeist has to be pulling it from the metadata which I will grab new branch a zg fts and see if I can get it to spit out all the things it is indexing; hopefully with the translations.

I also checked with the Software Center and for "语" it found Orca but not gbrainy which is slightly weird. "通" in the Software Center finds Sudoku also, so I'm thinking it has to do with the metadata. If just one of the words in the metadata contain the CJK char then it would pull up as a hit (If the search has only one char), so maybe something is lost/gained during translation form English -> CJK. Not sure how translations are handled.

So I'm going to have to go through zg to find out everything it uses for indexing and then I'll see if I can find where in the meta data theses CJK chars are for the irrelevant applications.

Additional test result from https://bugs.launchpad.net/oem-priority/+bug/745243/comments/29 (see screenshot attachment)

Test Case
=======

User want to find a application named as "terminal" (終端機 in Chinese), this Chinese word "終端機" has 3 characters , first character is "終", second is "端", the third is "機".

Step To Reproduce:

1. type "終端" unity2d search entry

Excepted Result:

2. the application "終端機" shows in result (User can find what application he wants)

Actually Result:

No result

Other Related Info:

* type "terminal" in unity2d search entry can find the application "終端機" (User can find what application he wants)
* type "終" unity2d search entry can find the application "終端機" (User can find what application he wants)
* type "端機" unity2d search entry can find the application "終端機", and "端機" is not actually a Chinese word, it does not mean anything here. (Users usually do not try "端機" keyword for finding "終端機" application)

Env
====

Ubuntu 11.10 Alpha i386 (201108010.1)
Unity-2d: 3.8.14.1-0ubuntu1
libxapian22: 1.2.5-2~ppa1
xapian-tools: 1.2.5-2~ppa1
LANGUAGE=zh_TW:en_US:en

This problem is located in the translation of the applications. The comment in the *.desktop file gets translated with those CJK chars.

Here is where the incorrect translation lies for gbrainy
http://bazaar.launchpad.net/~vcs-imports/gbrainy/master/view/head:/po/zh_CN.po#L715

Same for gnome-sudoku
http://bazaar.launchpad.net/~vcs-imports/gnome-games/master/view/head:/po/zh_CN.po#L5011

You can confirm it even more so if you copy that entire line and search it in the dash. ie. screen shot:
sudoku: http://imgur.com/U2s8D
gbrainy: http://imgur.com/Rf5dB